mxnet
|
Classes | |
class | Accuracy |
class | AdaDeltaOptimizer |
class | AdaGradOptimizer |
class | AdamOptimizer |
class | Bilinear |
class | Constant |
class | Context |
Context interface. More... | |
class | DataBatch |
Default object for holding a mini-batch of data and related information. More... | |
class | DataIter |
class | EvalMetric |
class | Executor |
Executor interface. More... | |
class | FactorScheduler |
class | FeedForward |
struct | FeedForwardConfig |
class | Initializer |
class | KVStore |
class | LogLoss |
class | LRScheduler |
lr scheduler interface More... | |
class | MAE |
class | Monitor |
Monitor interface. More... | |
class | MSE |
class | MSRAPrelu |
class | MXDataIter |
struct | MXDataIterBlob |
class | MXDataIterMap |
class | NDArray |
NDArray interface. More... | |
struct | NDBlob |
struct to store NDArrayHandle More... | |
class | Normal |
class | One |
class | Operator |
Operator interface. More... | |
class | OpMap |
OpMap instance holds a map of all the symbol creators so we can get symbol creators by name. This is used internally by Symbol and Operator. More... | |
class | Optimizer |
Optimizer interface. More... | |
class | OptimizerRegistry |
class | PSNR |
class | RMSE |
class | RMSPropOptimizer |
class | SGDOptimizer |
struct | Shape |
dynamic shape class that can hold shape of arbirary dimension More... | |
class | SignumOptimizer |
struct | SymBlob |
struct to store SymbolHandle More... | |
class | Symbol |
Symbol interface. More... | |
class | Uniform |
class | Xavier |
class | Zero |
Typedefs | |
typedef unsigned | index_t |
typedef std::function< Optimizer *()> | OptimizerCreator |
Enumerations | |
enum | OpReqType { kNullOp, kWriteTo, kWriteInplace, kAddTo } |
enum | DeviceType { kCPU = 1, kGPU = 2, kCPUPinned = 3 } |
enum | PickMode { PickMode::kClip = 0, PickMode::kWrap = 1 } |
Specify how out-of-bound indices behave. Default is "clip". "clip" means clip to the range. So, if all indices mentioned are too large, they are replaced by the index that addresses the last element along an axis. "wrap" means to wrap. More... | |
enum | DotForwardStype { DotForwardStype::kNone = 0, DotForwardStype::kCsr = 1, DotForwardStype::kDefault = 2, DotForwardStype::kRow_sparse = 3 } |
The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still produce an output of the. More... | |
enum | Batch_dotForwardStype { Batch_dotForwardStype::kNone = 0, Batch_dotForwardStype::kCsr = 1, Batch_dotForwardStype::kDefault = 2, Batch_dotForwardStype::kRow_sparse = 3 } |
The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still produce an output of the. More... | |
enum | CastDtype { CastDtype::kFloat16 = 0, CastDtype::kFloat32 = 1, CastDtype::kFloat64 = 2, CastDtype::kInt32 = 3, CastDtype::kInt64 = 4, CastDtype::kInt8 = 5, CastDtype::kUint8 = 6 } |
Output data type. More... | |
enum | Amp_castDtype { Amp_castDtype::kFloat16 = 0, Amp_castDtype::kFloat32 = 1, Amp_castDtype::kFloat64 = 2, Amp_castDtype::kInt32 = 3, Amp_castDtype::kInt64 = 4, Amp_castDtype::kInt8 = 5, Amp_castDtype::kUint8 = 6 } |
Output data type. More... | |
enum | TopkRetTyp { TopkRetTyp::kBoth = 0, TopkRetTyp::kIndices = 1, TopkRetTyp::kMask = 2, TopkRetTyp::kValue = 3 } |
The return type. "value" means to return the top k values, "indices" means to return the indices of the top k values, "mask" means to return a mask array containing 0 and 1. 1 means the top k values. "both" means to return a list of both values and. More... | |
enum | TopkDtype { TopkDtype::kFloat16 = 0, TopkDtype::kFloat32 = 1, TopkDtype::kFloat64 = 2, TopkDtype::kInt32 = 3, TopkDtype::kUint8 = 4 } |
DType of the output indices when ret_typ is "indices" or "both". An error will. More... | |
enum | ArgsortDtype { ArgsortDtype::kFloat16 = 0, ArgsortDtype::kFloat32 = 1, ArgsortDtype::kFloat64 = 2, ArgsortDtype::kInt32 = 3, ArgsortDtype::kUint8 = 4 } |
DType of the output indices. It is only valid when ret_typ is "indices" or "both". An error will be raised if the selected data type cannot precisely. More... | |
enum | EmbeddingDtype { EmbeddingDtype::kFloat16 = 0, EmbeddingDtype::kFloat32 = 1, EmbeddingDtype::kFloat64 = 2, EmbeddingDtype::kInt32 = 3, EmbeddingDtype::kInt64 = 4, EmbeddingDtype::kInt8 = 5, EmbeddingDtype::kUint8 = 6 } |
Data type of weight. More... | |
enum | TakeMode { TakeMode::kClip = 0, TakeMode::kRaise = 1, TakeMode::kWrap = 2 } |
Specify how out-of-bound indices bahave. Default is "clip". "clip" means clip to the range. So, if all indices mentioned are too large, they are replaced by the index that addresses the last element along an axis. "wrap" means to wrap. More... | |
enum | One_hotDtype { One_hotDtype::kFloat16 = 0, One_hotDtype::kFloat32 = 1, One_hotDtype::kFloat64 = 2, One_hotDtype::kInt32 = 3, One_hotDtype::kInt64 = 4, One_hotDtype::kInt8 = 5, One_hotDtype::kUint8 = 6 } |
DType of the output. More... | |
enum | Cast_storageStype { Cast_storageStype::kCsr = 0, Cast_storageStype::kDefault = 1, Cast_storageStype::kRow_sparse = 2 } |
Output storage type. More... | |
enum | NormOutDtype { NormOutDtype::kNone = 0, NormOutDtype::kFloat16 = 1, NormOutDtype::kFloat32 = 2, NormOutDtype::kFloat64 = 3, NormOutDtype::kInt32 = 4, NormOutDtype::kInt64 = 5, NormOutDtype::kInt8 = 6 } |
The data type of the output. More... | |
enum | PoolingPoolType { PoolingPoolType::kAvg = 0, PoolingPoolType::kLp = 1, PoolingPoolType::kMax = 2, PoolingPoolType::kSum = 3 } |
Pooling type to be applied. More... | |
enum | PoolingPoolingConvention { PoolingPoolingConvention::kFull = 0, PoolingPoolingConvention::kSame = 1, PoolingPoolingConvention::kValid = 2 } |
Pooling convention to be applied. More... | |
enum | PoolingLayout { PoolingLayout::kNone = 0, PoolingLayout::kNCDHW = 1, PoolingLayout::kNCHW = 2, PoolingLayout::kNCW = 3, PoolingLayout::kNDHWC = 4, PoolingLayout::kNHWC = 5, PoolingLayout::kNWC = 6 } |
Set layout for input and output. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d. More... | |
enum | SoftmaxDtype { SoftmaxDtype::kNone = 0, SoftmaxDtype::kFloat16 = 1, SoftmaxDtype::kFloat32 = 2, SoftmaxDtype::kFloat64 = 3 } |
DType of the output in case this can't be inferred. Defaults to the same as. More... | |
enum | SoftminDtype { SoftminDtype::kNone = 0, SoftminDtype::kFloat16 = 1, SoftminDtype::kFloat32 = 2, SoftminDtype::kFloat64 = 3 } |
DType of the output in case this can't be inferred. Defaults to the same as. More... | |
enum | Log_softmaxDtype { Log_softmaxDtype::kNone = 0, Log_softmaxDtype::kFloat16 = 1, Log_softmaxDtype::kFloat32 = 2, Log_softmaxDtype::kFloat64 = 3 } |
DType of the output in case this can't be inferred. Defaults to the same as. More... | |
enum | DeconvolutionCudnnTune { DeconvolutionCudnnTune::kNone = 0, DeconvolutionCudnnTune::kFastest = 1, DeconvolutionCudnnTune::kLimited_workspace = 2, DeconvolutionCudnnTune::kOff = 3 } |
Whether to pick convolution algorithm by running performance test. More... | |
enum | DeconvolutionLayout { DeconvolutionLayout::kNone = 0, DeconvolutionLayout::kNCDHW = 1, DeconvolutionLayout::kNCHW = 2, DeconvolutionLayout::kNCW = 3, DeconvolutionLayout::kNDHWC = 4, DeconvolutionLayout::kNHWC = 5 } |
Set layout for input, output and weight. Empty for default layout, NCW for 1d,. More... | |
enum | ActivationActType { ActivationActType::kRelu = 0, ActivationActType::kSigmoid = 1, ActivationActType::kSoftrelu = 2, ActivationActType::kSoftsign = 3, ActivationActType::kTanh = 4 } |
Activation function to be applied. More... | |
enum | CTCLossBlankLabel { CTCLossBlankLabel::kFirst = 0, CTCLossBlankLabel::kLast = 1 } |
Set the label that is reserved for blank label.If "first", 0-th label is reserved, and label values for tokens in the vocabulary are between 1 and alphabet_size-1 , and the padding mask is -1 . If "last", last label value alphabet_size-1 is reserved for blank label instead, and label values for tokens in the vocabulary are between 0 and alphabet_size-2 , and the. More... | |
enum | ConvolutionCudnnTune { ConvolutionCudnnTune::kNone = 0, ConvolutionCudnnTune::kFastest = 1, ConvolutionCudnnTune::kLimited_workspace = 2, ConvolutionCudnnTune::kOff = 3 } |
Whether to pick convolution algo by running performance test. More... | |
enum | ConvolutionLayout { ConvolutionLayout::kNone = 0, ConvolutionLayout::kNCDHW = 1, ConvolutionLayout::kNCHW = 2, ConvolutionLayout::kNCW = 3, ConvolutionLayout::kNDHWC = 4, ConvolutionLayout::kNHWC = 5 } |
Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d.NHWC and NDHWC are. More... | |
enum | UpSamplingSampleType { UpSamplingSampleType::kBilinear = 0, UpSamplingSampleType::kNearest = 1 } |
upsampling method More... | |
enum | UpSamplingMultiInputMode { UpSamplingMultiInputMode::kConcat = 0, UpSamplingMultiInputMode::kSum = 1 } |
How to handle multiple input. concat means concatenate upsampled images along the channel dimension. sum means add all images together, only available for. More... | |
enum | DropoutMode { DropoutMode::kAlways = 0, DropoutMode::kTraining = 1 } |
Whether to only turn on dropout during training or to also turn on for. More... | |
enum | SoftmaxActivationMode { SoftmaxActivationMode::kChannel = 0, SoftmaxActivationMode::kInstance = 1 } |
Specifies how to compute the softmax. If set to instance , it computes softmax for each instance. If set to channel , It computes cross channel. More... | |
enum | LeakyReLUActType { LeakyReLUActType::kElu = 0, LeakyReLUActType::kGelu = 1, LeakyReLUActType::kLeaky = 2, LeakyReLUActType::kPrelu = 3, LeakyReLUActType::kRrelu = 4, LeakyReLUActType::kSelu = 5 } |
Activation function to be applied. More... | |
enum | RNNMode { RNNMode::kGru = 0, RNNMode::kLstm = 1, RNNMode::kRnn_relu = 2, RNNMode::kRnn_tanh = 3 } |
the type of RNN to compute More... | |
enum | SoftmaxOutputNormalization { SoftmaxOutputNormalization::kBatch = 0, SoftmaxOutputNormalization::kNull = 1, SoftmaxOutputNormalization::kValid = 2 } |
Normalizes the gradient. More... | |
enum | PadMode { PadMode::kConstant = 0, PadMode::kEdge = 1, PadMode::kReflect = 2 } |
Padding type to use. "constant" pads with constant_value "edge" pads using the edge values of the input array "reflect" pads by reflecting values with. More... | |
enum | GridGeneratorTransformType { GridGeneratorTransformType::kAffine = 0, GridGeneratorTransformType::kWarp = 1 } |
The type of transformation. For affine , input data should be an affine matrix of size (batch, 6). For warp , input data should be an optical flow of size. More... | |
enum | Pooling_v1PoolType { Pooling_v1PoolType::kAvg = 0, Pooling_v1PoolType::kMax = 1, Pooling_v1PoolType::kSum = 2 } |
Pooling type to be applied. More... | |
enum | Pooling_v1PoolingConvention { Pooling_v1PoolingConvention::kFull = 0, Pooling_v1PoolingConvention::kValid = 1 } |
Pooling convention to be applied. More... | |
enum | Convolution_v1CudnnTune { Convolution_v1CudnnTune::kNone = 0, Convolution_v1CudnnTune::kFastest = 1, Convolution_v1CudnnTune::kLimited_workspace = 2, Convolution_v1CudnnTune::kOff = 3 } |
Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: 'off': no tuning 'limited_workspace': run test and pick the fastest algorithm that doesn't 'fastest': pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest. More... | |
enum | Convolution_v1Layout { Convolution_v1Layout::kNone = 0, Convolution_v1Layout::kNCDHW = 1, Convolution_v1Layout::kNCHW = 2, Convolution_v1Layout::kNDHWC = 3, Convolution_v1Layout::kNHWC = 4 } |
Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d. More... | |
enum | SpatialTransformerTransformType { SpatialTransformerTransformType::kAffine = 0 } |
transformation type More... | |
enum | SpatialTransformerSamplerType { SpatialTransformerSamplerType::kBilinear = 0 } |
sampling type More... | |
enum | L2NormalizationMode { L2NormalizationMode::kChannel = 0, L2NormalizationMode::kInstance = 1, L2NormalizationMode::kSpatial = 2 } |
Specify the dimension along which to compute L2 norm. More... | |
enum | MakeLossNormalization { MakeLossNormalization::kBatch = 0, MakeLossNormalization::kNull = 1, MakeLossNormalization::kValid = 2 } |
If this is set to null, the output gradient will not be normalized. If this is set to batch, the output gradient will be divided by the batch size. If this is set to valid, the output gradient will be divided by the number of valid input. More... | |
Functions | |
NDArray | _default_monitor_func (const NDArray &x) |
Default function for monitor that computes statistics of the input tensor, which is the mean absolute |x|/size(x) More... | |
std::ostream & | operator<< (std::ostream &out, const NDArray &ndarray) |
Symbol | khatri_rao (const std::string &symbol_name, const std::vector< Symbol > &args) |
Computes the Khatri-Rao product of the input matrices. More... | |
Symbol | all_finite (const std::string &symbol_name, Symbol data, bool init_output=true) |
Check if all the float numbers in the array are finite (used for AMP) More... | |
Symbol | multi_all_finite (const std::string &symbol_name, const std::vector< Symbol > &data, int num_arrays=1, bool init_output=true) |
Check if all the float numbers in all the arrays are finite (used for AMP) More... | |
Symbol | Custom (const std::string &symbol_name, const std::vector< Symbol > &data, const std::string &op_type) |
Apply a custom operator implemented in a frontend language (like Python). More... | |
Symbol | broadcast_power (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns result of first array elements raised to powers from second array,. More... | |
Symbol | broadcast_maximum (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise maximum of the input arrays with broadcasting. More... | |
Symbol | broadcast_minimum (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise minimum of the input arrays with broadcasting. More... | |
Symbol | broadcast_hypot (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the hypotenuse of a right angled triangle, given its "legs" with broadcasting. More... | |
Symbol | Reshape (const std::string &symbol_name, Symbol data, Shape shape=Shape(), bool reverse=false, Shape target_shape=Shape(), bool keep_highest=false) |
Reshapes the input array. More... | |
Symbol | Flatten (const std::string &symbol_name, Symbol data) |
Flattens the input array into a 2-D array by collapsing the higher dimensions. More... | |
Symbol | transpose (const std::string &symbol_name, Symbol data, Shape axes=Shape()) |
Permutes the dimensions of an array. More... | |
Symbol | expand_dims (const std::string &symbol_name, Symbol data, int axis) |
Inserts a new axis of size 1 into the array shape. More... | |
Symbol | slice (const std::string &symbol_name, Symbol data, Shape begin, Shape end, Shape step=Shape()) |
Slices a region of the array. More... | |
Symbol | slice_axis (const std::string &symbol_name, Symbol data, int axis, int begin, dmlc::optional< int > end) |
Slices along a given axis. More... | |
Symbol | slice_like (const std::string &symbol_name, Symbol data, Symbol shape_like, Shape axes=Shape()) |
Slices a region of the array like the shape of another array. More... | |
Symbol | clip (const std::string &symbol_name, Symbol data, mx_float a_min, mx_float a_max) |
Clips (limits) the values in an array. More... | |
Symbol | repeat (const std::string &symbol_name, Symbol data, int repeats, dmlc::optional< int > axis=dmlc::optional< int >()) |
Repeats elements of an array. More... | |
Symbol | tile (const std::string &symbol_name, Symbol data, Shape reps) |
Repeats the whole array multiple times. More... | |
Symbol | reverse (const std::string &symbol_name, Symbol data, Shape axis) |
Reverses the order of elements along given axis while preserving array shape. More... | |
Symbol | stack (const std::string &symbol_name, const std::vector< Symbol > &data, int num_args, int axis=0) |
Join a sequence of arrays along a new axis. More... | |
Symbol | squeeze (const std::string &symbol_name, const std::vector< Symbol > &data, dmlc::optional< Shape > axis=dmlc::optional< Shape >()) |
Remove single-dimensional entries from the shape of an array. Same behavior of defining the output tensor shape as numpy.squeeze for the most See the following note for exception. More... | |
Symbol | depth_to_space (const std::string &symbol_name, Symbol data, int block_size) |
Rearranges(permutes) data from depth into blocks of spatial data. Similar to ONNX DepthToSpace operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#DepthToSpace. The output is a new tensor where the values from depth dimension are moved in to height and width dimension. The reverse of this operation is. More... | |
Symbol | space_to_depth (const std::string &symbol_name, Symbol data, int block_size) |
Rearranges(permutes) blocks of spatial data into depth. Similar to ONNX SpaceToDepth operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#SpaceToDepth. More... | |
Symbol | zeros_like (const std::string &symbol_name, Symbol data) |
Return an array of zeros with the same shape, type and storage type as the input array. More... | |
Symbol | ones_like (const std::string &symbol_name, Symbol data) |
Return an array of ones with the same shape and type as the input array. More... | |
Symbol | add_n (const std::string &symbol_name, const std::vector< Symbol > &args) |
Adds all input arguments element-wise. More... | |
Symbol | argmax (const std::string &symbol_name, Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(), bool keepdims=false) |
Returns indices of the maximum values along an axis. More... | |
Symbol | argmin (const std::string &symbol_name, Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(), bool keepdims=false) |
Returns indices of the minimum values along an axis. More... | |
Symbol | argmax_channel (const std::string &symbol_name, Symbol data) |
Returns argmax indices of each channel from the input array. More... | |
Symbol | pick (const std::string &symbol_name, Symbol data, Symbol index, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool keepdims=false, PickMode mode=PickMode::kClip) |
Picks elements from an input array according to the input indices along the. More... | |
Symbol | dot (const std::string &symbol_name, Symbol lhs, Symbol rhs, bool transpose_a=false, bool transpose_b=false, DotForwardStype forward_stype=DotForwardStype::kNone) |
Dot product of two arrays. More... | |
Symbol | batch_dot (const std::string &symbol_name, Symbol lhs, Symbol rhs, bool transpose_a=false, bool transpose_b=false, Batch_dotForwardStype forward_stype=Batch_dotForwardStype::kNone) |
Batchwise dot product. More... | |
Symbol | broadcast_add (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise sum of the input arrays with broadcasting. More... | |
Symbol | broadcast_sub (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise difference of the input arrays with broadcasting. More... | |
Symbol | broadcast_mul (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise product of the input arrays with broadcasting. More... | |
Symbol | broadcast_div (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise division of the input arrays with broadcasting. More... | |
Symbol | broadcast_mod (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns element-wise modulo of the input arrays with broadcasting. More... | |
Symbol | relu (const std::string &symbol_name, Symbol data) |
Computes rectified linear activation. More... | |
Symbol | sigmoid (const std::string &symbol_name, Symbol data) |
Computes sigmoid of x element-wise. More... | |
Symbol | hard_sigmoid (const std::string &symbol_name, Symbol data, mx_float alpha=0.200000003, mx_float beta=0.5) |
Computes hard sigmoid of x element-wise. More... | |
Symbol | softsign (const std::string &symbol_name, Symbol data) |
Computes softsign of x element-wise. More... | |
Symbol | BlockGrad (const std::string &symbol_name, Symbol data) |
Stops gradient computation. More... | |
Symbol | make_loss (const std::string &symbol_name, Symbol data) |
Make your own loss function in network construction. More... | |
Symbol | reshape_like (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Reshape some or all dimensions of lhs to have the same shape as some or all. More... | |
Symbol | shape_array (const std::string &symbol_name, Symbol data, dmlc::optional< int > lhs_begin=dmlc::optional< int >(), dmlc::optional< int > lhs_end=dmlc::optional< int >(), dmlc::optional< int > rhs_begin=dmlc::optional< int >(), dmlc::optional< int > rhs_end=dmlc::optional< int >()) |
Returns a 1D int64 array containing the shape of data. More... | |
Symbol | size_array (const std::string &symbol_name, Symbol data) |
Returns a 1D int64 array containing the size of data. More... | |
Symbol | Cast (const std::string &symbol_name, Symbol data, CastDtype dtype) |
Casts all elements of the input to a new type. More... | |
Symbol | negative (const std::string &symbol_name, Symbol data) |
Numerical negative of the argument, element-wise. More... | |
Symbol | reciprocal (const std::string &symbol_name, Symbol data) |
Returns the reciprocal of the argument, element-wise. More... | |
Symbol | abs (const std::string &symbol_name, Symbol data) |
Returns element-wise absolute value of the input. More... | |
Symbol | sign (const std::string &symbol_name, Symbol data) |
Returns element-wise sign of the input. More... | |
Symbol | round (const std::string &symbol_name, Symbol data) |
Returns element-wise rounded value to the nearest integer of the input. More... | |
Symbol | rint (const std::string &symbol_name, Symbol data) |
Returns element-wise rounded value to the nearest integer of the input. More... | |
Symbol | ceil (const std::string &symbol_name, Symbol data) |
Returns element-wise ceiling of the input. More... | |
Symbol | floor (const std::string &symbol_name, Symbol data) |
Returns element-wise floor of the input. More... | |
Symbol | trunc (const std::string &symbol_name, Symbol data) |
Return the element-wise truncated value of the input. More... | |
Symbol | fix (const std::string &symbol_name, Symbol data) |
Returns element-wise rounded value to the nearest \ integer towards zero of the input. More... | |
Symbol | square (const std::string &symbol_name, Symbol data) |
Returns element-wise squared value of the input. More... | |
Symbol | sqrt (const std::string &symbol_name, Symbol data) |
Returns element-wise square-root value of the input. More... | |
Symbol | rsqrt (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse square-root value of the input. More... | |
Symbol | cbrt (const std::string &symbol_name, Symbol data) |
Returns element-wise cube-root value of the input. More... | |
Symbol | erf (const std::string &symbol_name, Symbol data) |
Returns element-wise gauss error function of the input. More... | |
Symbol | erfinv (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse gauss error function of the input. More... | |
Symbol | rcbrt (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse cube-root value of the input. More... | |
Symbol | exp (const std::string &symbol_name, Symbol data) |
Returns element-wise exponential value of the input. More... | |
Symbol | log (const std::string &symbol_name, Symbol data) |
Returns element-wise Natural logarithmic value of the input. More... | |
Symbol | log10 (const std::string &symbol_name, Symbol data) |
Returns element-wise Base-10 logarithmic value of the input. More... | |
Symbol | log2 (const std::string &symbol_name, Symbol data) |
Returns element-wise Base-2 logarithmic value of the input. More... | |
Symbol | log1p (const std::string &symbol_name, Symbol data) |
Returns element-wise log(1 + x) value of the input. More... | |
Symbol | expm1 (const std::string &symbol_name, Symbol data) |
Returns exp(x) - 1 computed element-wise on the input. More... | |
Symbol | gamma (const std::string &symbol_name, Symbol data) |
Returns the gamma function (extension of the factorial function \ to the reals), computed element-wise on the input array. More... | |
Symbol | gammaln (const std::string &symbol_name, Symbol data) |
Returns element-wise log of the absolute value of the gamma function \ of the input. More... | |
Symbol | logical_not (const std::string &symbol_name, Symbol data) |
Returns the result of logical NOT (!) function. More... | |
Symbol | amp_cast (const std::string &symbol_name, Symbol data, Amp_castDtype dtype) |
Cast function between low precision float/FP32 used by AMP. More... | |
Symbol | amp_multicast (const std::string &symbol_name, const std::vector< Symbol > &data, int num_outputs) |
Cast function used by AMP, that casts its inputs to the common widest type. More... | |
Symbol | topk (const std::string &symbol_name, Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), int k=1, TopkRetTyp ret_typ=TopkRetTyp::kIndices, bool is_ascend=false, TopkDtype dtype=TopkDtype::kFloat32) |
Returns the top k elements in an input array along the given axis. The returned elements will be sorted. More... | |
Symbol | sort (const std::string &symbol_name, Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool is_ascend=true) |
Returns a sorted copy of an input array along the given axis. More... | |
Symbol | argsort (const std::string &symbol_name, Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool is_ascend=true, ArgsortDtype dtype=ArgsortDtype::kFloat32) |
Returns the indices that would sort an input array along the given axis. More... | |
Symbol | elemwise_add (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Adds arguments element-wise. More... | |
Symbol | elemwise_sub (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Subtracts arguments element-wise. More... | |
Symbol | elemwise_mul (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Multiplies arguments element-wise. More... | |
Symbol | elemwise_div (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Divides arguments element-wise. More... | |
Symbol | Embedding (const std::string &symbol_name, Symbol data, Symbol weight, int input_dim, int output_dim, EmbeddingDtype dtype=EmbeddingDtype::kFloat32, bool sparse_grad=false) |
Maps integer indices to vector representations (embeddings). More... | |
Symbol | take (const std::string &symbol_name, Symbol a, Symbol indices, int axis=0, TakeMode mode=TakeMode::kClip) |
Takes elements from an input array along the given axis. More... | |
Symbol | batch_take (const std::string &symbol_name, Symbol a, Symbol indices) |
Takes elements from a data batch. More... | |
Symbol | one_hot (const std::string &symbol_name, Symbol indices, int depth, double on_value=1, double off_value=0, One_hotDtype dtype=One_hotDtype::kFloat32) |
Returns a one-hot array. More... | |
Symbol | gather_nd (const std::string &symbol_name, Symbol data, Symbol indices) |
Gather elements or slices from data and store to a tensor whose shape is defined by indices . More... | |
Symbol | scatter_nd (const std::string &symbol_name, Symbol data, Symbol indices, Shape shape) |
Scatters data into a new tensor according to indices. More... | |
Symbol | broadcast_equal (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise equal to (==) comparison operation with. More... | |
Symbol | broadcast_not_equal (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise not equal to (!=) comparison operation. More... | |
Symbol | broadcast_greater (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise greater than (>) comparison operation. More... | |
Symbol | broadcast_greater_equal (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise greater than or equal to (>=) comparison. More... | |
Symbol | broadcast_lesser (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise lesser than (<) comparison operation. More... | |
Symbol | broadcast_lesser_equal (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise lesser than or equal to (<=) comparison. More... | |
Symbol | broadcast_logical_and (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical and with broadcasting. More... | |
Symbol | broadcast_logical_or (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical or with broadcasting. More... | |
Symbol | broadcast_logical_xor (const std::string &symbol_name, Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical xor with broadcasting. More... | |
Symbol | diag (const std::string &symbol_name, Symbol data, int k=0, int axis1=0, int axis2=1) |
Extracts a diagonal or constructs a diagonal array. More... | |
Symbol | where (const std::string &symbol_name, Symbol condition, Symbol x, Symbol y) |
Return the elements, either from x or y, depending on the condition. More... | |
Symbol | smooth_l1 (const std::string &symbol_name, Symbol data, mx_float scalar) |
Calculate Smooth L1 Loss(lhs, scalar) by summing. More... | |
Symbol | cast_storage (const std::string &symbol_name, Symbol data, Cast_storageStype stype) |
Casts tensor storage type to the new type. More... | |
Symbol | sum (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the sum of array elements over given axes. More... | |
Symbol | mean (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the mean of array elements over given axes. More... | |
Symbol | prod (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the product of array elements over given axes. More... | |
Symbol | nansum (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the sum of array elements over given axes treating Not a Numbers. More... | |
Symbol | nanprod (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the product of array elements over given axes treating Not a Numbers. More... | |
Symbol | max (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the max of array elements over given axes. More... | |
Symbol | min (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the min of array elements over given axes. More... | |
Symbol | broadcast_axis (const std::string &symbol_name, Symbol data, Shape axis=Shape(), Shape size=Shape()) |
Broadcasts the input array over particular axes. More... | |
Symbol | broadcast_to (const std::string &symbol_name, Symbol data, Shape shape=Shape()) |
Broadcasts the input array to a new shape. More... | |
Symbol | broadcast_like (const std::string &symbol_name, Symbol lhs, Symbol rhs, dmlc::optional< Shape > lhs_axes=dmlc::optional< Shape >(), dmlc::optional< Shape > rhs_axes=dmlc::optional< Shape >()) |
Broadcasts lhs to have the same shape as rhs. More... | |
Symbol | norm (const std::string &symbol_name, Symbol data, int ord=2, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), NormOutDtype out_dtype=NormOutDtype::kNone, bool keepdims=false) |
Computes the norm on an NDArray. More... | |
Symbol | sin (const std::string &symbol_name, Symbol data) |
Computes the element-wise sine of the input array. More... | |
Symbol | cos (const std::string &symbol_name, Symbol data) |
Computes the element-wise cosine of the input array. More... | |
Symbol | tan (const std::string &symbol_name, Symbol data) |
Computes the element-wise tangent of the input array. More... | |
Symbol | arcsin (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse sine of the input array. More... | |
Symbol | arccos (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse cosine of the input array. More... | |
Symbol | arctan (const std::string &symbol_name, Symbol data) |
Returns element-wise inverse tangent of the input array. More... | |
Symbol | degrees (const std::string &symbol_name, Symbol data) |
Converts each element of the input array from radians to degrees. More... | |
Symbol | radians (const std::string &symbol_name, Symbol data) |
Converts each element of the input array from degrees to radians. More... | |
Symbol | sinh (const std::string &symbol_name, Symbol data) |
Returns the hyperbolic sine of the input array, computed element-wise. More... | |
Symbol | cosh (const std::string &symbol_name, Symbol data) |
Returns the hyperbolic cosine of the input array, computed element-wise. More... | |
Symbol | tanh (const std::string &symbol_name, Symbol data) |
Returns the hyperbolic tangent of the input array, computed element-wise. More... | |
Symbol | arcsinh (const std::string &symbol_name, Symbol data) |
Returns the element-wise inverse hyperbolic sine of the input array, \ computed element-wise. More... | |
Symbol | arccosh (const std::string &symbol_name, Symbol data) |
Returns the element-wise inverse hyperbolic cosine of the input array, \ computed element-wise. More... | |
Symbol | arctanh (const std::string &symbol_name, Symbol data) |
Returns the element-wise inverse hyperbolic tangent of the input array, \ computed element-wise. More... | |
Symbol | Pooling (const std::string &symbol_name, Symbol data, Shape kernel=Shape(), PoolingPoolType pool_type=PoolingPoolType::kMax, bool global_pool=false, bool cudnn_off=false, PoolingPoolingConvention pooling_convention=PoolingPoolingConvention::kValid, Shape stride=Shape(), Shape pad=Shape(), dmlc::optional< int > p_value=dmlc::optional< int >(), dmlc::optional< bool > count_include_pad=dmlc::optional< bool >(), PoolingLayout layout=PoolingLayout::kNone) |
Performs pooling on the input. More... | |
Symbol | softmax (const std::string &symbol_name, Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), SoftmaxDtype dtype=SoftmaxDtype::kNone) |
Applies the softmax function. More... | |
Symbol | softmin (const std::string &symbol_name, Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), SoftminDtype dtype=SoftminDtype::kNone) |
Applies the softmin function. More... | |
Symbol | log_softmax (const std::string &symbol_name, Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), Log_softmaxDtype dtype=Log_softmaxDtype::kNone) |
Computes the log softmax of the input. This is equivalent to computing softmax followed by log. More... | |
Symbol | Deconvolution (const std::string &symbol_name, Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), Shape adj=Shape(), Shape target_shape=Shape(), uint32_t num_group=1, uint64_t workspace=512, bool no_bias=true, DeconvolutionCudnnTune cudnn_tune=DeconvolutionCudnnTune::kNone, bool cudnn_off=false, DeconvolutionLayout layout=DeconvolutionLayout::kNone) |
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller. More... | |
Symbol | Activation (const std::string &symbol_name, Symbol data, ActivationActType act_type) |
Applies an activation function element-wise to the input. More... | |
Symbol | BatchNorm (const std::string &symbol_name, Symbol data, Symbol gamma, Symbol beta, Symbol moving_mean, Symbol moving_var, double eps=0.0010000000474974513, mx_float momentum=0.899999976, bool fix_gamma=true, bool use_global_stats=false, bool output_mean_var=false, int axis=1, bool cudnn_off=false) |
Batch normalization. More... | |
Symbol | CTCLoss (const std::string &symbol_name, Symbol data, Symbol label, Symbol data_lengths, Symbol label_lengths, bool use_data_lengths=false, bool use_label_lengths=false, CTCLossBlankLabel blank_label=CTCLossBlankLabel::kFirst) |
Connectionist Temporal Classification Loss. More... | |
Symbol | FullyConnected (const std::string &symbol_name, Symbol data, Symbol weight, Symbol bias, int num_hidden, bool no_bias=false, bool flatten=true) |
Applies a linear transformation: :math:Y = XW^T + b . More... | |
Symbol | Convolution (const std::string &symbol_name, Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), uint32_t num_group=1, uint64_t workspace=1024, bool no_bias=false, ConvolutionCudnnTune cudnn_tune=ConvolutionCudnnTune::kNone, bool cudnn_off=false, ConvolutionLayout layout=ConvolutionLayout::kNone) |
Compute N-D convolution on *(N+2)*-D input. More... | |
Symbol | UpSampling (const std::string &symbol_name, const std::vector< Symbol > &data, int scale, UpSamplingSampleType sample_type, int num_args, int num_filter=0, UpSamplingMultiInputMode multi_input_mode=UpSamplingMultiInputMode::kConcat, uint64_t workspace=512) |
Upsamples the given input data. More... | |
Symbol | Concat (const std::string &symbol_name, const std::vector< Symbol > &data, int num_args, int dim=1) |
Joins input arrays along a given axis. More... | |
Symbol | LayerNorm (const std::string &symbol_name, Symbol data, Symbol gamma, Symbol beta, int axis=-1, mx_float eps=9.99999975e-06, bool output_mean_var=false) |
Layer normalization. More... | |
Symbol | LRN (const std::string &symbol_name, Symbol data, uint32_t nsize, mx_float alpha=9.99999975e-05, mx_float beta=0.75, mx_float knorm=2) |
Applies local response normalization to the input. More... | |
Symbol | Dropout (const std::string &symbol_name, Symbol data, mx_float p=0.5, DropoutMode mode=DropoutMode::kTraining, Shape axes=Shape(), dmlc::optional< bool > cudnn_off=dmlc::optional< bool >(0)) |
Applies dropout operation to input array. More... | |
Symbol | SoftmaxActivation (const std::string &symbol_name, Symbol data, SoftmaxActivationMode mode=SoftmaxActivationMode::kInstance) |
Applies softmax activation to input. This is intended for internal layers. More... | |
Symbol | moments (const std::string &symbol_name, Symbol data, dmlc::optional< Shape > axes=dmlc::optional< Shape >(), bool keepdims=false) |
Calculate the mean and variance of data . More... | |
Symbol | LeakyReLU (const std::string &symbol_name, Symbol data, Symbol gamma, LeakyReLUActType act_type=LeakyReLUActType::kLeaky, mx_float slope=0.25, mx_float lower_bound=0.125, mx_float upper_bound=0.333999991) |
Applies Leaky rectified linear unit activation element-wise to the input. More... | |
Symbol | RNN (const std::string &symbol_name, Symbol data, Symbol parameters, Symbol state, Symbol state_cell, Symbol sequence_length, uint32_t state_size, uint32_t num_layers, RNNMode mode, bool bidirectional=false, mx_float p=0, bool state_outputs=false, dmlc::optional< int > projection_size=dmlc::optional< int >(), dmlc::optional< double > lstm_state_clip_min=dmlc::optional< double >(), dmlc::optional< double > lstm_state_clip_max=dmlc::optional< double >(), bool lstm_state_clip_nan=false, bool use_sequence_length=false) |
Applies recurrent layers to input data. Currently, vanilla RNN, LSTM and GRU are implemented, with both multi-layer and bidirectional support. More... | |
Symbol | SoftmaxOutput (const std::string &symbol_name, Symbol data, Symbol label, mx_float grad_scale=1, mx_float ignore_label=-1, bool multi_output=false, bool use_ignore=false, bool preserve_shape=false, SoftmaxOutputNormalization normalization=SoftmaxOutputNormalization::kNull, bool out_grad=false, mx_float smooth_alpha=0) |
Computes the gradient of cross entropy loss with respect to softmax output. More... | |
Symbol | SwapAxis (const std::string &symbol_name, Symbol data, uint32_t dim1=0, uint32_t dim2=0) |
Interchanges two axes of an array. More... | |
Symbol | BatchNorm_v1 (const std::string &symbol_name, Symbol data, Symbol gamma, Symbol beta, mx_float eps=0.00100000005, mx_float momentum=0.899999976, bool fix_gamma=true, bool use_global_stats=false, bool output_mean_var=false) |
Batch normalization. More... | |
Symbol | softmax_cross_entropy (const std::string &symbol_name, Symbol data, Symbol label) |
Calculate cross entropy of softmax output and one-hot label. More... | |
Symbol | LinearRegressionOutput (const std::string &symbol_name, Symbol data, Symbol label, mx_float grad_scale=1) |
Computes and optimizes for squared loss during backward propagation. Just outputs data during forward propagation. More... | |
Symbol | MAERegressionOutput (const std::string &symbol_name, Symbol data, Symbol label, mx_float grad_scale=1) |
Computes mean absolute error of the input. More... | |
Symbol | LogisticRegressionOutput (const std::string &symbol_name, Symbol data, Symbol label, mx_float grad_scale=1) |
Applies a logistic function to the input. More... | |
Symbol | IdentityAttachKLSparseReg (const std::string &symbol_name, Symbol data, mx_float sparseness_target=0.100000001, mx_float penalty=0.00100000005, mx_float momentum=0.899999976) |
Apply a sparse regularization to the output a sigmoid activation function. More... | |
Symbol | signsgd_update (const std::string &symbol_name, Symbol weight, Symbol grad, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for SignSGD optimizer. More... | |
Symbol | signum_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float wd_lh=0) |
SIGN momentUM (Signum) optimizer. More... | |
Symbol | multi_sgd_update (const std::string &symbol_name, const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Update function for Stochastic Gradient Descent (SDG) optimizer. More... | |
Symbol | multi_sgd_mom_update (const std::string &symbol_name, const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float momentum=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Momentum update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | multi_mp_sgd_update (const std::string &symbol_name, const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer. More... | |
Symbol | multi_mp_sgd_mom_update (const std::string &symbol_name, const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float momentum=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Momentum update function for multi-precision Stochastic Gradient Descent (SGD) More... | |
Symbol | sgd_update (const std::string &symbol_name, Symbol weight, Symbol grad, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | sgd_mom_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Momentum update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | mp_sgd_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol weight32, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Updater function for multi-precision sgd optimizer. More... | |
Symbol | mp_sgd_mom_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mom, Symbol weight32, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Updater function for multi-precision sgd optimizer. More... | |
Symbol | ftml_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol d, Symbol v, Symbol z, mx_float lr, int t, mx_float beta1=0.600000024, mx_float beta2=0.999000013, double epsilon=9.9999999392252903e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_grad=-1) |
The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf. More... | |
Symbol | adam_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mean, Symbol var, mx_float lr, mx_float beta1=0.899999976, mx_float beta2=0.999000013, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Update function for Adam optimizer. Adam is seen as a generalization of AdaGrad. More... | |
Symbol | nag_mom_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for Nesterov Accelerated Gradient( NAG) optimizer. It updates the weights using the following formula,. More... | |
Symbol | mp_nag_mom_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol mom, Symbol weight32, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for multi-precision Nesterov Accelerated Gradient( NAG) More... | |
Symbol | rmsprop_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol n, mx_float lr, mx_float gamma1=0.949999988, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float clip_weights=-1) |
Update function for RMSProp optimizer. More... | |
Symbol | rmspropalex_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol n, Symbol g, Symbol delta, mx_float lr, mx_float gamma1=0.949999988, mx_float gamma2=0.899999976, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float clip_weights=-1) |
Update function for RMSPropAlex optimizer. More... | |
Symbol | ftrl_update (const std::string &symbol_name, Symbol weight, Symbol grad, Symbol z, Symbol n, mx_float lr, mx_float lamda1=0.00999999978, mx_float beta=1, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for Ftrl optimizer. Referenced from Ad Click Prediction: a View from the Trenches, available at http://dl.acm.org/citation.cfm?id=2488200. More... | |
Symbol | SliceChannel (const std::string &symbol_name, Symbol data, int num_outputs, int axis=1, bool squeeze_axis=false) |
Splits an array along a particular axis into multiple sub-arrays. More... | |
Symbol | Pad (const std::string &symbol_name, Symbol data, PadMode mode, Shape pad_width, double constant_value=0) |
Pads an input array with a constant or edge values of the array. More... | |
Symbol | InstanceNorm (const std::string &symbol_name, Symbol data, Symbol gamma, Symbol beta, mx_float eps=0.00100000005) |
Applies instance normalization to the n-dimensional input array. More... | |
Symbol | GridGenerator (const std::string &symbol_name, Symbol data, GridGeneratorTransformType transform_type, Shape target_shape=Shape(0, 0)) |
Generates 2D sampling grid for bilinear sampling. More... | |
Symbol | Pooling_v1 (const std::string &symbol_name, Symbol data, Shape kernel=Shape(), Pooling_v1PoolType pool_type=Pooling_v1PoolType::kMax, bool global_pool=false, Pooling_v1PoolingConvention pooling_convention=Pooling_v1PoolingConvention::kValid, Shape stride=Shape(), Shape pad=Shape()) |
This operator is DEPRECATED. Perform pooling on the input. More... | |
Symbol | Convolution_v1 (const std::string &symbol_name, Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), uint32_t num_group=1, uint64_t workspace=1024, bool no_bias=false, Convolution_v1CudnnTune cudnn_tune=Convolution_v1CudnnTune::kNone, bool cudnn_off=false, Convolution_v1Layout layout=Convolution_v1Layout::kNone) |
This operator is DEPRECATED. Apply convolution to input then add a bias. More... | |
Symbol | Crop (const std::string &symbol_name, const std::vector< Symbol > &data, int num_args, Shape offset=Shape(0, 0), Shape h_w=Shape(0, 0), bool center_crop=false) |
.. note:: Crop is deprecated. Use slice instead. More... | |
Symbol | SequenceReverse (const std::string &symbol_name, Symbol data, Symbol sequence_length, bool use_sequence_length=false, int axis=0) |
Reverses the elements of each sequence. More... | |
Symbol | SpatialTransformer (const std::string &symbol_name, Symbol data, Symbol loc, SpatialTransformerTransformType transform_type, SpatialTransformerSamplerType sampler_type, Shape target_shape=Shape(0, 0), dmlc::optional< bool > cudnn_off=dmlc::optional< bool >()) |
Applies a spatial transformer to input feature map. More... | |
Symbol | BilinearSampler (const std::string &symbol_name, Symbol data, Symbol grid, dmlc::optional< bool > cudnn_off=dmlc::optional< bool >()) |
Applies bilinear sampling to input feature map. More... | |
Symbol | ROIPooling (const std::string &symbol_name, Symbol data, Symbol rois, Shape pooled_size, mx_float spatial_scale) |
Performs region of interest(ROI) pooling on the input array. More... | |
Symbol | SequenceLast (const std::string &symbol_name, Symbol data, Symbol sequence_length, bool use_sequence_length=false, int axis=0) |
Takes the last element of a sequence. More... | |
Symbol | L2Normalization (const std::string &symbol_name, Symbol data, mx_float eps=1.00000001e-10, L2NormalizationMode mode=L2NormalizationMode::kInstance) |
Normalize the input array using the L2 norm. More... | |
Symbol | MakeLoss (const std::string &symbol_name, Symbol data, mx_float grad_scale=1, mx_float valid_thresh=0, MakeLossNormalization normalization=MakeLossNormalization::kNull) |
Make your own loss function in network construction. More... | |
Symbol | SVMOutput (const std::string &symbol_name, Symbol data, Symbol label, mx_float margin=1, mx_float regularization_coefficient=1, bool use_linear=false) |
Computes support vector machine based transformation of the input. More... | |
Symbol | Correlation (const std::string &symbol_name, Symbol data1, Symbol data2, uint32_t kernel_size=1, uint32_t max_displacement=1, uint32_t stride1=1, uint32_t stride2=1, uint32_t pad_size=0, bool is_multiply=true) |
Applies correlation to inputs. More... | |
Symbol | SequenceMask (const std::string &symbol_name, Symbol data, Symbol sequence_length, bool use_sequence_length=false, mx_float value=0, int axis=0) |
Sets all elements outside the sequence to a constant value. More... | |
Symbol | fill_element_0index (const std::string &symbol_name, Symbol lhs, Symbol mhs, Symbol rhs) |
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function. More... | |
Symbol | khatri_rao (const std::vector< Symbol > &args) |
Computes the Khatri-Rao product of the input matrices. More... | |
Symbol | all_finite (Symbol data, bool init_output=true) |
Check if all the float numbers in the array are finite (used for AMP) More... | |
Symbol | multi_all_finite (const std::vector< Symbol > &data, int num_arrays=1, bool init_output=true) |
Check if all the float numbers in all the arrays are finite (used for AMP) More... | |
Symbol | Custom (const std::vector< Symbol > &data, const std::string &op_type) |
Apply a custom operator implemented in a frontend language (like Python). More... | |
Symbol | broadcast_power (Symbol lhs, Symbol rhs) |
Returns result of first array elements raised to powers from second array,. More... | |
Symbol | broadcast_maximum (Symbol lhs, Symbol rhs) |
Returns element-wise maximum of the input arrays with broadcasting. More... | |
Symbol | broadcast_minimum (Symbol lhs, Symbol rhs) |
Returns element-wise minimum of the input arrays with broadcasting. More... | |
Symbol | broadcast_hypot (Symbol lhs, Symbol rhs) |
Returns the hypotenuse of a right angled triangle, given its "legs" with broadcasting. More... | |
Symbol | Reshape (Symbol data, Shape shape=Shape(), bool reverse=false, Shape target_shape=Shape(), bool keep_highest=false) |
Reshapes the input array. More... | |
Symbol | Flatten (Symbol data) |
Flattens the input array into a 2-D array by collapsing the higher dimensions. More... | |
Symbol | transpose (Symbol data, Shape axes=Shape()) |
Permutes the dimensions of an array. More... | |
Symbol | expand_dims (Symbol data, int axis) |
Inserts a new axis of size 1 into the array shape. More... | |
Symbol | slice (Symbol data, Shape begin, Shape end, Shape step=Shape()) |
Slices a region of the array. More... | |
Symbol | slice_axis (Symbol data, int axis, int begin, dmlc::optional< int > end) |
Slices along a given axis. More... | |
Symbol | slice_like (Symbol data, Symbol shape_like, Shape axes=Shape()) |
Slices a region of the array like the shape of another array. More... | |
Symbol | clip (Symbol data, mx_float a_min, mx_float a_max) |
Clips (limits) the values in an array. More... | |
Symbol | repeat (Symbol data, int repeats, dmlc::optional< int > axis=dmlc::optional< int >()) |
Repeats elements of an array. More... | |
Symbol | tile (Symbol data, Shape reps) |
Repeats the whole array multiple times. More... | |
Symbol | reverse (Symbol data, Shape axis) |
Reverses the order of elements along given axis while preserving array shape. More... | |
Symbol | stack (const std::vector< Symbol > &data, int num_args, int axis=0) |
Join a sequence of arrays along a new axis. More... | |
Symbol | squeeze (const std::vector< Symbol > &data, dmlc::optional< Shape > axis=dmlc::optional< Shape >()) |
Remove single-dimensional entries from the shape of an array. Same behavior of defining the output tensor shape as numpy.squeeze for the most See the following note for exception. More... | |
Symbol | depth_to_space (Symbol data, int block_size) |
Rearranges(permutes) data from depth into blocks of spatial data. Similar to ONNX DepthToSpace operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#DepthToSpace. The output is a new tensor where the values from depth dimension are moved in to height and width dimension. The reverse of this operation is. More... | |
Symbol | space_to_depth (Symbol data, int block_size) |
Rearranges(permutes) blocks of spatial data into depth. Similar to ONNX SpaceToDepth operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#SpaceToDepth. More... | |
Symbol | zeros_like (Symbol data) |
Return an array of zeros with the same shape, type and storage type as the input array. More... | |
Symbol | ones_like (Symbol data) |
Return an array of ones with the same shape and type as the input array. More... | |
Symbol | add_n (const std::vector< Symbol > &args) |
Adds all input arguments element-wise. More... | |
Symbol | argmax (Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(), bool keepdims=false) |
Returns indices of the maximum values along an axis. More... | |
Symbol | argmin (Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(), bool keepdims=false) |
Returns indices of the minimum values along an axis. More... | |
Symbol | argmax_channel (Symbol data) |
Returns argmax indices of each channel from the input array. More... | |
Symbol | pick (Symbol data, Symbol index, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool keepdims=false, PickMode mode=PickMode::kClip) |
Picks elements from an input array according to the input indices along the. More... | |
Symbol | dot (Symbol lhs, Symbol rhs, bool transpose_a=false, bool transpose_b=false, DotForwardStype forward_stype=DotForwardStype::kNone) |
Dot product of two arrays. More... | |
Symbol | batch_dot (Symbol lhs, Symbol rhs, bool transpose_a=false, bool transpose_b=false, Batch_dotForwardStype forward_stype=Batch_dotForwardStype::kNone) |
Batchwise dot product. More... | |
Symbol | broadcast_add (Symbol lhs, Symbol rhs) |
Returns element-wise sum of the input arrays with broadcasting. More... | |
Symbol | broadcast_sub (Symbol lhs, Symbol rhs) |
Returns element-wise difference of the input arrays with broadcasting. More... | |
Symbol | broadcast_mul (Symbol lhs, Symbol rhs) |
Returns element-wise product of the input arrays with broadcasting. More... | |
Symbol | broadcast_div (Symbol lhs, Symbol rhs) |
Returns element-wise division of the input arrays with broadcasting. More... | |
Symbol | broadcast_mod (Symbol lhs, Symbol rhs) |
Returns element-wise modulo of the input arrays with broadcasting. More... | |
Symbol | relu (Symbol data) |
Computes rectified linear activation. More... | |
Symbol | sigmoid (Symbol data) |
Computes sigmoid of x element-wise. More... | |
Symbol | hard_sigmoid (Symbol data, mx_float alpha=0.200000003, mx_float beta=0.5) |
Computes hard sigmoid of x element-wise. More... | |
Symbol | softsign (Symbol data) |
Computes softsign of x element-wise. More... | |
Symbol | BlockGrad (Symbol data) |
Stops gradient computation. More... | |
Symbol | make_loss (Symbol data) |
Make your own loss function in network construction. More... | |
Symbol | reshape_like (Symbol lhs, Symbol rhs) |
Reshape some or all dimensions of lhs to have the same shape as some or all. More... | |
Symbol | shape_array (Symbol data, dmlc::optional< int > lhs_begin=dmlc::optional< int >(), dmlc::optional< int > lhs_end=dmlc::optional< int >(), dmlc::optional< int > rhs_begin=dmlc::optional< int >(), dmlc::optional< int > rhs_end=dmlc::optional< int >()) |
Returns a 1D int64 array containing the shape of data. More... | |
Symbol | size_array (Symbol data) |
Returns a 1D int64 array containing the size of data. More... | |
Symbol | Cast (Symbol data, CastDtype dtype) |
Casts all elements of the input to a new type. More... | |
Symbol | negative (Symbol data) |
Numerical negative of the argument, element-wise. More... | |
Symbol | reciprocal (Symbol data) |
Returns the reciprocal of the argument, element-wise. More... | |
Symbol | abs (Symbol data) |
Returns element-wise absolute value of the input. More... | |
Symbol | sign (Symbol data) |
Returns element-wise sign of the input. More... | |
Symbol | round (Symbol data) |
Returns element-wise rounded value to the nearest integer of the input. More... | |
Symbol | rint (Symbol data) |
Returns element-wise rounded value to the nearest integer of the input. More... | |
Symbol | ceil (Symbol data) |
Returns element-wise ceiling of the input. More... | |
Symbol | floor (Symbol data) |
Returns element-wise floor of the input. More... | |
Symbol | trunc (Symbol data) |
Return the element-wise truncated value of the input. More... | |
Symbol | fix (Symbol data) |
Returns element-wise rounded value to the nearest \ integer towards zero of the input. More... | |
Symbol | square (Symbol data) |
Returns element-wise squared value of the input. More... | |
Symbol | sqrt (Symbol data) |
Returns element-wise square-root value of the input. More... | |
Symbol | rsqrt (Symbol data) |
Returns element-wise inverse square-root value of the input. More... | |
Symbol | cbrt (Symbol data) |
Returns element-wise cube-root value of the input. More... | |
Symbol | erf (Symbol data) |
Returns element-wise gauss error function of the input. More... | |
Symbol | erfinv (Symbol data) |
Returns element-wise inverse gauss error function of the input. More... | |
Symbol | rcbrt (Symbol data) |
Returns element-wise inverse cube-root value of the input. More... | |
Symbol | exp (Symbol data) |
Returns element-wise exponential value of the input. More... | |
Symbol | log (Symbol data) |
Returns element-wise Natural logarithmic value of the input. More... | |
Symbol | log10 (Symbol data) |
Returns element-wise Base-10 logarithmic value of the input. More... | |
Symbol | log2 (Symbol data) |
Returns element-wise Base-2 logarithmic value of the input. More... | |
Symbol | log1p (Symbol data) |
Returns element-wise log(1 + x) value of the input. More... | |
Symbol | expm1 (Symbol data) |
Returns exp(x) - 1 computed element-wise on the input. More... | |
Symbol | gamma (Symbol data) |
Returns the gamma function (extension of the factorial function \ to the reals), computed element-wise on the input array. More... | |
Symbol | gammaln (Symbol data) |
Returns element-wise log of the absolute value of the gamma function \ of the input. More... | |
Symbol | logical_not (Symbol data) |
Returns the result of logical NOT (!) function. More... | |
Symbol | amp_cast (Symbol data, Amp_castDtype dtype) |
Cast function between low precision float/FP32 used by AMP. More... | |
Symbol | amp_multicast (const std::vector< Symbol > &data, int num_outputs) |
Cast function used by AMP, that casts its inputs to the common widest type. More... | |
Symbol | topk (Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), int k=1, TopkRetTyp ret_typ=TopkRetTyp::kIndices, bool is_ascend=false, TopkDtype dtype=TopkDtype::kFloat32) |
Returns the top k elements in an input array along the given axis. The returned elements will be sorted. More... | |
Symbol | sort (Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool is_ascend=true) |
Returns a sorted copy of an input array along the given axis. More... | |
Symbol | argsort (Symbol data, dmlc::optional< int > axis=dmlc::optional< int >(-1), bool is_ascend=true, ArgsortDtype dtype=ArgsortDtype::kFloat32) |
Returns the indices that would sort an input array along the given axis. More... | |
Symbol | elemwise_add (Symbol lhs, Symbol rhs) |
Adds arguments element-wise. More... | |
Symbol | elemwise_sub (Symbol lhs, Symbol rhs) |
Subtracts arguments element-wise. More... | |
Symbol | elemwise_mul (Symbol lhs, Symbol rhs) |
Multiplies arguments element-wise. More... | |
Symbol | elemwise_div (Symbol lhs, Symbol rhs) |
Divides arguments element-wise. More... | |
Symbol | Embedding (Symbol data, Symbol weight, int input_dim, int output_dim, EmbeddingDtype dtype=EmbeddingDtype::kFloat32, bool sparse_grad=false) |
Maps integer indices to vector representations (embeddings). More... | |
Symbol | take (Symbol a, Symbol indices, int axis=0, TakeMode mode=TakeMode::kClip) |
Takes elements from an input array along the given axis. More... | |
Symbol | batch_take (Symbol a, Symbol indices) |
Takes elements from a data batch. More... | |
Symbol | one_hot (Symbol indices, int depth, double on_value=1, double off_value=0, One_hotDtype dtype=One_hotDtype::kFloat32) |
Returns a one-hot array. More... | |
Symbol | gather_nd (Symbol data, Symbol indices) |
Gather elements or slices from data and store to a tensor whose shape is defined by indices . More... | |
Symbol | scatter_nd (Symbol data, Symbol indices, Shape shape) |
Scatters data into a new tensor according to indices. More... | |
Symbol | broadcast_equal (Symbol lhs, Symbol rhs) |
Returns the result of element-wise equal to (==) comparison operation with. More... | |
Symbol | broadcast_not_equal (Symbol lhs, Symbol rhs) |
Returns the result of element-wise not equal to (!=) comparison operation. More... | |
Symbol | broadcast_greater (Symbol lhs, Symbol rhs) |
Returns the result of element-wise greater than (>) comparison operation. More... | |
Symbol | broadcast_greater_equal (Symbol lhs, Symbol rhs) |
Returns the result of element-wise greater than or equal to (>=) comparison. More... | |
Symbol | broadcast_lesser (Symbol lhs, Symbol rhs) |
Returns the result of element-wise lesser than (<) comparison operation. More... | |
Symbol | broadcast_lesser_equal (Symbol lhs, Symbol rhs) |
Returns the result of element-wise lesser than or equal to (<=) comparison. More... | |
Symbol | broadcast_logical_and (Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical and with broadcasting. More... | |
Symbol | broadcast_logical_or (Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical or with broadcasting. More... | |
Symbol | broadcast_logical_xor (Symbol lhs, Symbol rhs) |
Returns the result of element-wise logical xor with broadcasting. More... | |
Symbol | diag (Symbol data, int k=0, int axis1=0, int axis2=1) |
Extracts a diagonal or constructs a diagonal array. More... | |
Symbol | where (Symbol condition, Symbol x, Symbol y) |
Return the elements, either from x or y, depending on the condition. More... | |
Symbol | smooth_l1 (Symbol data, mx_float scalar) |
Calculate Smooth L1 Loss(lhs, scalar) by summing. More... | |
Symbol | cast_storage (Symbol data, Cast_storageStype stype) |
Casts tensor storage type to the new type. More... | |
Symbol | sum (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the sum of array elements over given axes. More... | |
Symbol | mean (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the mean of array elements over given axes. More... | |
Symbol | prod (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the product of array elements over given axes. More... | |
Symbol | nansum (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the sum of array elements over given axes treating Not a Numbers. More... | |
Symbol | nanprod (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the product of array elements over given axes treating Not a Numbers. More... | |
Symbol | max (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the max of array elements over given axes. More... | |
Symbol | min (Symbol data, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), bool keepdims=false, bool exclude=false) |
Computes the min of array elements over given axes. More... | |
Symbol | broadcast_axis (Symbol data, Shape axis=Shape(), Shape size=Shape()) |
Broadcasts the input array over particular axes. More... | |
Symbol | broadcast_to (Symbol data, Shape shape=Shape()) |
Broadcasts the input array to a new shape. More... | |
Symbol | broadcast_like (Symbol lhs, Symbol rhs, dmlc::optional< Shape > lhs_axes=dmlc::optional< Shape >(), dmlc::optional< Shape > rhs_axes=dmlc::optional< Shape >()) |
Broadcasts lhs to have the same shape as rhs. More... | |
Symbol | norm (Symbol data, int ord=2, dmlc::optional< Shape > axis=dmlc::optional< Shape >(), NormOutDtype out_dtype=NormOutDtype::kNone, bool keepdims=false) |
Computes the norm on an NDArray. More... | |
Symbol | sin (Symbol data) |
Computes the element-wise sine of the input array. More... | |
Symbol | cos (Symbol data) |
Computes the element-wise cosine of the input array. More... | |
Symbol | tan (Symbol data) |
Computes the element-wise tangent of the input array. More... | |
Symbol | arcsin (Symbol data) |
Returns element-wise inverse sine of the input array. More... | |
Symbol | arccos (Symbol data) |
Returns element-wise inverse cosine of the input array. More... | |
Symbol | arctan (Symbol data) |
Returns element-wise inverse tangent of the input array. More... | |
Symbol | degrees (Symbol data) |
Converts each element of the input array from radians to degrees. More... | |
Symbol | radians (Symbol data) |
Converts each element of the input array from degrees to radians. More... | |
Symbol | sinh (Symbol data) |
Returns the hyperbolic sine of the input array, computed element-wise. More... | |
Symbol | cosh (Symbol data) |
Returns the hyperbolic cosine of the input array, computed element-wise. More... | |
Symbol | tanh (Symbol data) |
Returns the hyperbolic tangent of the input array, computed element-wise. More... | |
Symbol | arcsinh (Symbol data) |
Returns the element-wise inverse hyperbolic sine of the input array, \ computed element-wise. More... | |
Symbol | arccosh (Symbol data) |
Returns the element-wise inverse hyperbolic cosine of the input array, \ computed element-wise. More... | |
Symbol | arctanh (Symbol data) |
Returns the element-wise inverse hyperbolic tangent of the input array, \ computed element-wise. More... | |
Symbol | Pooling (Symbol data, Shape kernel=Shape(), PoolingPoolType pool_type=PoolingPoolType::kMax, bool global_pool=false, bool cudnn_off=false, PoolingPoolingConvention pooling_convention=PoolingPoolingConvention::kValid, Shape stride=Shape(), Shape pad=Shape(), dmlc::optional< int > p_value=dmlc::optional< int >(), dmlc::optional< bool > count_include_pad=dmlc::optional< bool >(), PoolingLayout layout=PoolingLayout::kNone) |
Performs pooling on the input. More... | |
Symbol | softmax (Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), SoftmaxDtype dtype=SoftmaxDtype::kNone) |
Applies the softmax function. More... | |
Symbol | softmin (Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), SoftminDtype dtype=SoftminDtype::kNone) |
Applies the softmin function. More... | |
Symbol | log_softmax (Symbol data, int axis=-1, dmlc::optional< double > temperature=dmlc::optional< double >(), Log_softmaxDtype dtype=Log_softmaxDtype::kNone) |
Computes the log softmax of the input. This is equivalent to computing softmax followed by log. More... | |
Symbol | Deconvolution (Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), Shape adj=Shape(), Shape target_shape=Shape(), uint32_t num_group=1, uint64_t workspace=512, bool no_bias=true, DeconvolutionCudnnTune cudnn_tune=DeconvolutionCudnnTune::kNone, bool cudnn_off=false, DeconvolutionLayout layout=DeconvolutionLayout::kNone) |
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller. More... | |
Symbol | Activation (Symbol data, ActivationActType act_type) |
Applies an activation function element-wise to the input. More... | |
Symbol | BatchNorm (Symbol data, Symbol gamma, Symbol beta, Symbol moving_mean, Symbol moving_var, double eps=0.0010000000474974513, mx_float momentum=0.899999976, bool fix_gamma=true, bool use_global_stats=false, bool output_mean_var=false, int axis=1, bool cudnn_off=false) |
Batch normalization. More... | |
Symbol | CTCLoss (Symbol data, Symbol label, Symbol data_lengths, Symbol label_lengths, bool use_data_lengths=false, bool use_label_lengths=false, CTCLossBlankLabel blank_label=CTCLossBlankLabel::kFirst) |
Connectionist Temporal Classification Loss. More... | |
Symbol | FullyConnected (Symbol data, Symbol weight, Symbol bias, int num_hidden, bool no_bias=false, bool flatten=true) |
Applies a linear transformation: :math:Y = XW^T + b . More... | |
Symbol | Convolution (Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), uint32_t num_group=1, uint64_t workspace=1024, bool no_bias=false, ConvolutionCudnnTune cudnn_tune=ConvolutionCudnnTune::kNone, bool cudnn_off=false, ConvolutionLayout layout=ConvolutionLayout::kNone) |
Compute N-D convolution on *(N+2)*-D input. More... | |
Symbol | UpSampling (const std::vector< Symbol > &data, int scale, UpSamplingSampleType sample_type, int num_args, int num_filter=0, UpSamplingMultiInputMode multi_input_mode=UpSamplingMultiInputMode::kConcat, uint64_t workspace=512) |
Upsamples the given input data. More... | |
Symbol | Concat (const std::vector< Symbol > &data, int num_args, int dim=1) |
Joins input arrays along a given axis. More... | |
Symbol | LayerNorm (Symbol data, Symbol gamma, Symbol beta, int axis=-1, mx_float eps=9.99999975e-06, bool output_mean_var=false) |
Layer normalization. More... | |
Symbol | LRN (Symbol data, uint32_t nsize, mx_float alpha=9.99999975e-05, mx_float beta=0.75, mx_float knorm=2) |
Applies local response normalization to the input. More... | |
Symbol | Dropout (Symbol data, mx_float p=0.5, DropoutMode mode=DropoutMode::kTraining, Shape axes=Shape(), dmlc::optional< bool > cudnn_off=dmlc::optional< bool >(0)) |
Applies dropout operation to input array. More... | |
Symbol | SoftmaxActivation (Symbol data, SoftmaxActivationMode mode=SoftmaxActivationMode::kInstance) |
Applies softmax activation to input. This is intended for internal layers. More... | |
Symbol | moments (Symbol data, dmlc::optional< Shape > axes=dmlc::optional< Shape >(), bool keepdims=false) |
Calculate the mean and variance of data . More... | |
Symbol | LeakyReLU (Symbol data, Symbol gamma, LeakyReLUActType act_type=LeakyReLUActType::kLeaky, mx_float slope=0.25, mx_float lower_bound=0.125, mx_float upper_bound=0.333999991) |
Applies Leaky rectified linear unit activation element-wise to the input. More... | |
Symbol | RNN (Symbol data, Symbol parameters, Symbol state, Symbol state_cell, Symbol sequence_length, uint32_t state_size, uint32_t num_layers, RNNMode mode, bool bidirectional=false, mx_float p=0, bool state_outputs=false, dmlc::optional< int > projection_size=dmlc::optional< int >(), dmlc::optional< double > lstm_state_clip_min=dmlc::optional< double >(), dmlc::optional< double > lstm_state_clip_max=dmlc::optional< double >(), bool lstm_state_clip_nan=false, bool use_sequence_length=false) |
Applies recurrent layers to input data. Currently, vanilla RNN, LSTM and GRU are implemented, with both multi-layer and bidirectional support. More... | |
Symbol | SoftmaxOutput (Symbol data, Symbol label, mx_float grad_scale=1, mx_float ignore_label=-1, bool multi_output=false, bool use_ignore=false, bool preserve_shape=false, SoftmaxOutputNormalization normalization=SoftmaxOutputNormalization::kNull, bool out_grad=false, mx_float smooth_alpha=0) |
Computes the gradient of cross entropy loss with respect to softmax output. More... | |
Symbol | SwapAxis (Symbol data, uint32_t dim1=0, uint32_t dim2=0) |
Interchanges two axes of an array. More... | |
Symbol | BatchNorm_v1 (Symbol data, Symbol gamma, Symbol beta, mx_float eps=0.00100000005, mx_float momentum=0.899999976, bool fix_gamma=true, bool use_global_stats=false, bool output_mean_var=false) |
Batch normalization. More... | |
Symbol | softmax_cross_entropy (Symbol data, Symbol label) |
Calculate cross entropy of softmax output and one-hot label. More... | |
Symbol | LinearRegressionOutput (Symbol data, Symbol label, mx_float grad_scale=1) |
Computes and optimizes for squared loss during backward propagation. Just outputs data during forward propagation. More... | |
Symbol | MAERegressionOutput (Symbol data, Symbol label, mx_float grad_scale=1) |
Computes mean absolute error of the input. More... | |
Symbol | LogisticRegressionOutput (Symbol data, Symbol label, mx_float grad_scale=1) |
Applies a logistic function to the input. More... | |
Symbol | IdentityAttachKLSparseReg (Symbol data, mx_float sparseness_target=0.100000001, mx_float penalty=0.00100000005, mx_float momentum=0.899999976) |
Apply a sparse regularization to the output a sigmoid activation function. More... | |
Symbol | signsgd_update (Symbol weight, Symbol grad, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for SignSGD optimizer. More... | |
Symbol | signum_update (Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float wd_lh=0) |
SIGN momentUM (Signum) optimizer. More... | |
Symbol | multi_sgd_update (const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Update function for Stochastic Gradient Descent (SDG) optimizer. More... | |
Symbol | multi_sgd_mom_update (const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float momentum=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Momentum update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | multi_mp_sgd_update (const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Update function for multi-precision Stochastic Gradient Descent (SDG) optimizer. More... | |
Symbol | multi_mp_sgd_mom_update (const std::vector< Symbol > &data, nnvm::Tuple< mx_float > lrs, nnvm::Tuple< mx_float > wds, mx_float momentum=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, int num_weights=1) |
Momentum update function for multi-precision Stochastic Gradient Descent (SGD) More... | |
Symbol | sgd_update (Symbol weight, Symbol grad, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | sgd_mom_update (Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Momentum update function for Stochastic Gradient Descent (SGD) optimizer. More... | |
Symbol | mp_sgd_update (Symbol weight, Symbol grad, Symbol weight32, mx_float lr, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Updater function for multi-precision sgd optimizer. More... | |
Symbol | mp_sgd_mom_update (Symbol weight, Symbol grad, Symbol mom, Symbol weight32, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Updater function for multi-precision sgd optimizer. More... | |
Symbol | ftml_update (Symbol weight, Symbol grad, Symbol d, Symbol v, Symbol z, mx_float lr, int t, mx_float beta1=0.600000024, mx_float beta2=0.999000013, double epsilon=9.9999999392252903e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_grad=-1) |
The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf. More... | |
Symbol | adam_update (Symbol weight, Symbol grad, Symbol mean, Symbol var, mx_float lr, mx_float beta1=0.899999976, mx_float beta2=0.999000013, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, bool lazy_update=true) |
Update function for Adam optimizer. Adam is seen as a generalization of AdaGrad. More... | |
Symbol | nag_mom_update (Symbol weight, Symbol grad, Symbol mom, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for Nesterov Accelerated Gradient( NAG) optimizer. It updates the weights using the following formula,. More... | |
Symbol | mp_nag_mom_update (Symbol weight, Symbol grad, Symbol mom, Symbol weight32, mx_float lr, mx_float momentum=0, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for multi-precision Nesterov Accelerated Gradient( NAG) More... | |
Symbol | rmsprop_update (Symbol weight, Symbol grad, Symbol n, mx_float lr, mx_float gamma1=0.949999988, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float clip_weights=-1) |
Update function for RMSProp optimizer. More... | |
Symbol | rmspropalex_update (Symbol weight, Symbol grad, Symbol n, Symbol g, Symbol delta, mx_float lr, mx_float gamma1=0.949999988, mx_float gamma2=0.899999976, mx_float epsilon=9.99999994e-09, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1, mx_float clip_weights=-1) |
Update function for RMSPropAlex optimizer. More... | |
Symbol | ftrl_update (Symbol weight, Symbol grad, Symbol z, Symbol n, mx_float lr, mx_float lamda1=0.00999999978, mx_float beta=1, mx_float wd=0, mx_float rescale_grad=1, mx_float clip_gradient=-1) |
Update function for Ftrl optimizer. Referenced from Ad Click Prediction: a View from the Trenches, available at http://dl.acm.org/citation.cfm?id=2488200. More... | |
Symbol | SliceChannel (Symbol data, int num_outputs, int axis=1, bool squeeze_axis=false) |
Splits an array along a particular axis into multiple sub-arrays. More... | |
Symbol | Pad (Symbol data, PadMode mode, Shape pad_width, double constant_value=0) |
Pads an input array with a constant or edge values of the array. More... | |
Symbol | InstanceNorm (Symbol data, Symbol gamma, Symbol beta, mx_float eps=0.00100000005) |
Applies instance normalization to the n-dimensional input array. More... | |
Symbol | GridGenerator (Symbol data, GridGeneratorTransformType transform_type, Shape target_shape=Shape(0, 0)) |
Generates 2D sampling grid for bilinear sampling. More... | |
Symbol | Pooling_v1 (Symbol data, Shape kernel=Shape(), Pooling_v1PoolType pool_type=Pooling_v1PoolType::kMax, bool global_pool=false, Pooling_v1PoolingConvention pooling_convention=Pooling_v1PoolingConvention::kValid, Shape stride=Shape(), Shape pad=Shape()) |
This operator is DEPRECATED. Perform pooling on the input. More... | |
Symbol | Convolution_v1 (Symbol data, Symbol weight, Symbol bias, Shape kernel, uint32_t num_filter, Shape stride=Shape(), Shape dilate=Shape(), Shape pad=Shape(), uint32_t num_group=1, uint64_t workspace=1024, bool no_bias=false, Convolution_v1CudnnTune cudnn_tune=Convolution_v1CudnnTune::kNone, bool cudnn_off=false, Convolution_v1Layout layout=Convolution_v1Layout::kNone) |
This operator is DEPRECATED. Apply convolution to input then add a bias. More... | |
Symbol | Crop (const std::vector< Symbol > &data, int num_args, Shape offset=Shape(0, 0), Shape h_w=Shape(0, 0), bool center_crop=false) |
.. note:: Crop is deprecated. Use slice instead. More... | |
Symbol | SequenceReverse (Symbol data, Symbol sequence_length, bool use_sequence_length=false, int axis=0) |
Reverses the elements of each sequence. More... | |
Symbol | SpatialTransformer (Symbol data, Symbol loc, SpatialTransformerTransformType transform_type, SpatialTransformerSamplerType sampler_type, Shape target_shape=Shape(0, 0), dmlc::optional< bool > cudnn_off=dmlc::optional< bool >()) |
Applies a spatial transformer to input feature map. More... | |
Symbol | BilinearSampler (Symbol data, Symbol grid, dmlc::optional< bool > cudnn_off=dmlc::optional< bool >()) |
Applies bilinear sampling to input feature map. More... | |
Symbol | ROIPooling (Symbol data, Symbol rois, Shape pooled_size, mx_float spatial_scale) |
Performs region of interest(ROI) pooling on the input array. More... | |
Symbol | SequenceLast (Symbol data, Symbol sequence_length, bool use_sequence_length=false, int axis=0) |
Takes the last element of a sequence. More... | |
Symbol | L2Normalization (Symbol data, mx_float eps=1.00000001e-10, L2NormalizationMode mode=L2NormalizationMode::kInstance) |
Normalize the input array using the L2 norm. More... | |
Symbol | MakeLoss (Symbol data, mx_float grad_scale=1, mx_float valid_thresh=0, MakeLossNormalization normalization=MakeLossNormalization::kNull) |
Make your own loss function in network construction. More... | |
Symbol | SVMOutput (Symbol data, Symbol label, mx_float margin=1, mx_float regularization_coefficient=1, bool use_linear=false) |
Computes support vector machine based transformation of the input. More... | |
Symbol | Correlation (Symbol data1, Symbol data2, uint32_t kernel_size=1, uint32_t max_displacement=1, uint32_t stride1=1, uint32_t stride2=1, uint32_t pad_size=0, bool is_multiply=true) |
Applies correlation to inputs. More... | |
Symbol | SequenceMask (Symbol data, Symbol sequence_length, bool use_sequence_length=false, mx_float value=0, int axis=0) |
Sets all elements outside the sequence to a constant value. More... | |
Symbol | fill_element_0index (Symbol lhs, Symbol mhs, Symbol rhs) |
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function. More... | |
Symbol | _Plus (Symbol lhs, Symbol rhs) |
Symbol | _Mul (Symbol lhs, Symbol rhs) |
Symbol | _Minus (Symbol lhs, Symbol rhs) |
Symbol | _Div (Symbol lhs, Symbol rhs) |
Symbol | _Mod (Symbol lhs, Symbol rhs) |
Symbol | _Power (Symbol lhs, Symbol rhs) |
Symbol | _Maximum (Symbol lhs, Symbol rhs) |
Symbol | _Minimum (Symbol lhs, Symbol rhs) |
Symbol | _PlusScalar (Symbol lhs, mx_float scalar) |
Symbol | _MinusScalar (Symbol lhs, mx_float scalar) |
Symbol | _RMinusScalar (mx_float scalar, Symbol rhs) |
Symbol | _MulScalar (Symbol lhs, mx_float scalar) |
Symbol | _DivScalar (Symbol lhs, mx_float scalar) |
Symbol | _RDivScalar (mx_float scalar, Symbol rhs) |
Symbol | _ModScalar (Symbol lhs, mx_float scalar) |
Symbol | _RModScalar (mx_float scalar, Symbol rhs) |
Symbol | _PowerScalar (Symbol lhs, mx_float scalar) |
Symbol | _RPowerScalar (mx_float scalar, Symbol rhs) |
Symbol | _MaximumScalar (Symbol lhs, mx_float scalar) |
Symbol | _MinimumScalar (Symbol lhs, mx_float scalar) |
Symbol | Crop (const std::string &symbol_name, int num_args, Symbol data, Symbol crop_like, Shape offset=Shape(0, 0), Shape h_w=Shape(0, 0), bool center_crop=false) |
Symbol | Activation (const std::string &symbol_name, Symbol data, const std::string &act_type) |
Apply activation function to input. Softmax Activation is only available with CUDNN on GPUand will be computed at each location across channel if input is 4D. More... | |
std::ostream & | operator<< (std::ostream &os, const Shape &shape) |
allow string printing of the shape More... | |
std::istream & | operator>> (std::istream &is, Shape &shape) |
read shape from the istream More... | |
Symbol | operator+ (mx_float lhs, const Symbol &rhs) |
Symbol | operator- (mx_float lhs, const Symbol &rhs) |
Symbol | operator* (mx_float lhs, const Symbol &rhs) |
Symbol | operator/ (mx_float lhs, const Symbol &rhs) |
Symbol | operator% (mx_float lhs, const Symbol &rhs) |
typedef unsigned mxnet::cpp::index_t |
typedef std::function<Optimizer*()> mxnet::cpp::OptimizerCreator |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: 'off': no tuning 'limited_workspace': run test and pick the fastest algorithm that doesn't 'fastest': pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest.
Enumerator | |
---|---|
kNone | |
kFastest | |
kLimited_workspace | |
kOff |
|
strong |
|
strong |
|
strong |
|
strong |
Set the label that is reserved for blank label.If "first", 0-th label is reserved, and label values for tokens in the vocabulary are between 1
and alphabet_size-1
, and the padding mask is -1
. If "last", last label value alphabet_size-1
is reserved for blank label instead, and label values for tokens in the vocabulary are between 0
and alphabet_size-2
, and the.
Enumerator | |
---|---|
kFirst | |
kLast |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
|
strong |
Default function for monitor that computes statistics of the input tensor, which is the mean absolute |x|/size(x)
x | The input tensor |
Returns element-wise absolute value of the input.
Example::
abs([-2, 0, 3]) = [2, 0, 3]
The storage type of abs
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L708
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise absolute value of the input.
Example::
abs([-2, 0, 3]) = [2, 0, 3]
The storage type of abs
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L708
data | The input array. |
|
inline |
Apply activation function to input. Softmax Activation is only available with CUDNN on GPUand will be computed at each location across channel if input is 4D.
symbol_name | name of the resulting symbol. |
data | Input data to activation function. |
act_type | Activation function to be applied. |
|
inline |
Applies an activation function element-wise to the input.
The following activation functions are supported:
relu
: Rectified Linear Unit, :math:y = max(x, 0)
sigmoid
: :math:y = \frac{1}{1 + exp(-x)}
tanh
: Hyperbolic tangent, :math:y = \frac{exp(x) - exp(-x)}{exp(x) + -
softrelu: Soft ReLU, or SoftPlus, :math:
y = log(1 + exp(x)) -
softsign: :math:
y = {x}{1 + abs(x)}`Defined in src/operator/nn/activation.cc:L167
symbol_name | name of the resulting symbol |
data | The input array. |
act_type | Activation function to be applied. |
|
inline |
Applies an activation function element-wise to the input.
The following activation functions are supported:
relu
: Rectified Linear Unit, :math:y = max(x, 0)
sigmoid
: :math:y = \frac{1}{1 + exp(-x)}
tanh
: Hyperbolic tangent, :math:y = \frac{exp(x) - exp(-x)}{exp(x) + -
softrelu: Soft ReLU, or SoftPlus, :math:
y = log(1 + exp(x)) -
softsign: :math:
y = {x}{1 + abs(x)}`Defined in src/operator/nn/activation.cc:L167
data | The input array. |
act_type | Activation function to be applied. |
|
inline |
Update function for Adam optimizer. Adam is seen as a generalization of AdaGrad.
Adam update consists of the following steps, where g represents gradient and m, are 1st and 2nd order moment estimates (mean and variance).
.. math::
g_t = J(W_{t-1})\ m_t = m_{t-1} + (1 - ) g_t\ v_t = v_{t-1} + (1 - ) g_t^2\ W_t = W_{t-1} - { m_t }{ { v_t } + }
It updates the weights using::
m = beta1*m + (1-beta1)*grad v = beta2*v + (1-beta2)*(grad**2) w += - learning_rate * m / (sqrt(v) + epsilon)
However, if grad's storage type is row_sparse
, lazy_update
is True and type of weight is the same as those of m and v, only the row slices whose indices appear in grad.indices are updated (for w, m
for row in grad.indices: m[row] = beta1*m[row] + (1-beta1)*grad[row] v[row] = beta2*v[row] + (1-beta2)*(grad[row]**2) w[row] += - learning_rate * m[row] / (sqrt(v[row]) + epsilon)
Defined in src/operator/optimizer_op.cc:L686
symbol_name | name of the resulting symbol |
weight | Weight |
grad | Gradient |
mean | Moving mean |
var | Moving variance |
lr | Learning rate |
beta1 | The decay rate for the 1st moment estimates. |
beta2 | The decay rate for the 2nd moment estimates. |
epsilon | A small constant for numerical stability. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_gradient | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
lazy_update | If true, lazy updates are applied if gradient's stype is row_sparse |
|
inline |
Update function for Adam optimizer. Adam is seen as a generalization of AdaGrad.
Adam update consists of the following steps, where g represents gradient and m, are 1st and 2nd order moment estimates (mean and variance).
.. math::
g_t = J(W_{t-1})\ m_t = m_{t-1} + (1 - ) g_t\ v_t = v_{t-1} + (1 - ) g_t^2\ W_t = W_{t-1} - { m_t }{ { v_t } + }
It updates the weights using::
m = beta1*m + (1-beta1)*grad v = beta2*v + (1-beta2)*(grad**2) w += - learning_rate * m / (sqrt(v) + epsilon)
However, if grad's storage type is row_sparse
, lazy_update
is True and type of weight is the same as those of m and v, only the row slices whose indices appear in grad.indices are updated (for w, m
for row in grad.indices: m[row] = beta1*m[row] + (1-beta1)*grad[row] v[row] = beta2*v[row] + (1-beta2)*(grad[row]**2) w[row] += - learning_rate * m[row] / (sqrt(v[row]) + epsilon)
Defined in src/operator/optimizer_op.cc:L686
weight | Weight |
grad | Gradient |
mean | Moving mean |
var | Moving variance |
lr | Learning rate |
beta1 | The decay rate for the 1st moment estimates. |
beta2 | The decay rate for the 2nd moment estimates. |
epsilon | A small constant for numerical stability. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_gradient | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
lazy_update | If true, lazy updates are applied if gradient's stype is row_sparse |
|
inline |
Adds all input arguments element-wise.
.. math:: add_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n
add_n
is potentially more efficient than calling add
by n
times.
The storage type of add_n
output depends on storage types of inputs
add_n
falls all inputs back to default storage and generatesDefined in src/operator/tensor/elemwise_sum.cc:L155
symbol_name | name of the resulting symbol |
args | Positional input arguments |
Adds all input arguments element-wise.
.. math:: add_n(a_1, a_2, ..., a_n) = a_1 + a_2 + ... + a_n
add_n
is potentially more efficient than calling add
by n
times.
The storage type of add_n
output depends on storage types of inputs
add_n
falls all inputs back to default storage and generatesDefined in src/operator/tensor/elemwise_sum.cc:L155
args | Positional input arguments |
|
inline |
Check if all the float numbers in the array are finite (used for AMP)
Defined in src/operator/contrib/all_finite.cc:L101
symbol_name | name of the resulting symbol |
data | Array |
init_output | Initialize output to 1. |
Check if all the float numbers in the array are finite (used for AMP)
Defined in src/operator/contrib/all_finite.cc:L101
data | Array |
init_output | Initialize output to 1. |
|
inline |
Cast function between low precision float/FP32 used by AMP.
It casts only between low precision float/FP32 and does not do anything for
Defined in src/operator/tensor/amp_cast.cc:L37
symbol_name | name of the resulting symbol |
data | The input. |
dtype | Output data type. |
|
inline |
Cast function between low precision float/FP32 used by AMP.
It casts only between low precision float/FP32 and does not do anything for
Defined in src/operator/tensor/amp_cast.cc:L37
data | The input. |
dtype | Output data type. |
|
inline |
Cast function used by AMP, that casts its inputs to the common widest type.
It casts only between low precision float/FP32 and does not do anything for
Defined in src/operator/tensor/amp_cast.cc:L71
symbol_name | name of the resulting symbol |
data | Weights |
num_outputs | Number of input/output pairs to be casted to the widest type. |
Cast function used by AMP, that casts its inputs to the common widest type.
It casts only between low precision float/FP32 and does not do anything for
Defined in src/operator/tensor/amp_cast.cc:L71
data | Weights |
num_outputs | Number of input/output pairs to be casted to the widest type. |
Returns element-wise inverse cosine of the input array.
The input should be in range [-1, 1]
. The output is in the closed interval :math:[0, \pi]
.. math:: arccos([-1, -.707, 0, .707, 1]) = [, 3/4, /2, /4, 0]
The storage type of arccos
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L179
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise inverse cosine of the input array.
The input should be in range [-1, 1]
. The output is in the closed interval :math:[0, \pi]
.. math:: arccos([-1, -.707, 0, .707, 1]) = [, 3/4, /2, /4, 0]
The storage type of arccos
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L179
data | The input array. |
Returns the element-wise inverse hyperbolic cosine of the input array, \ computed element-wise.
The storage type of arccosh
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L320
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the element-wise inverse hyperbolic cosine of the input array, \ computed element-wise.
The storage type of arccosh
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L320
data | The input array. |
Returns element-wise inverse sine of the input array.
The input should be in the range [-1, 1]
. The output is in the closed interval of [:math:-\pi/2
, :math:\pi/2
].
.. math:: arcsin([-1, -.707, 0, .707, 1]) = [-/2, -/4, 0, /4, /2]
The storage type of arcsin
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L160
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise inverse sine of the input array.
The input should be in the range [-1, 1]
. The output is in the closed interval of [:math:-\pi/2
, :math:\pi/2
].
.. math:: arcsin([-1, -.707, 0, .707, 1]) = [-/2, -/4, 0, /4, /2]
The storage type of arcsin
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L160
data | The input array. |
Returns the element-wise inverse hyperbolic sine of the input array, \ computed element-wise.
The storage type of arcsinh
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L306
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the element-wise inverse hyperbolic sine of the input array, \ computed element-wise.
The storage type of arcsinh
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L306
data | The input array. |
Returns element-wise inverse tangent of the input array.
The output is in the closed interval :math:[-\pi/2, \pi/2]
.. math:: arctan([-1, 0, 1]) = [-/4, 0, /4]
The storage type of arctan
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L200
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise inverse tangent of the input array.
The output is in the closed interval :math:[-\pi/2, \pi/2]
.. math:: arctan([-1, 0, 1]) = [-/4, 0, /4]
The storage type of arctan
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L200
data | The input array. |
Returns the element-wise inverse hyperbolic tangent of the input array, \ computed element-wise.
The storage type of arctanh
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L337
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the element-wise inverse hyperbolic tangent of the input array, \ computed element-wise.
The storage type of arctanh
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L337
data | The input array. |
|
inline |
Returns indices of the maximum values along an axis.
In the case of multiple occurrences of maximum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
// argmax along axis 0 argmax(x, axis=0) = [ 1., 1., 1.]
// argmax along axis 1 argmax(x, axis=1) = [ 2., 2.]
// argmax along axis 1 keeping same dims as an input array argmax(x, axis=1, keepdims=True) = [[ 2.], [ 2.]]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L52
symbol_name | name of the resulting symbol |
data | The input |
axis | The axis along which to perform the reduction. Negative values means indexing from right to left. `Requires axis to be set as int, because global \param keepdims If this is set to True`, the reduced axis is left in the result as |
|
inline |
Returns indices of the maximum values along an axis.
In the case of multiple occurrences of maximum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
// argmax along axis 0 argmax(x, axis=0) = [ 1., 1., 1.]
// argmax along axis 1 argmax(x, axis=1) = [ 2., 2.]
// argmax along axis 1 keeping same dims as an input array argmax(x, axis=1, keepdims=True) = [[ 2.], [ 2.]]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L52
data | The input |
axis | The axis along which to perform the reduction. Negative values means indexing from right to left. `Requires axis to be set as int, because global \param keepdims If this is set to True`, the reduced axis is left in the result as |
Returns argmax indices of each channel from the input array.
The result will be an NDArray of shape (num_channel,).
In case of multiple occurrences of the maximum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
argmax_channel(x) = [ 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L97
symbol_name | name of the resulting symbol |
data | The input array |
Returns argmax indices of each channel from the input array.
The result will be an NDArray of shape (num_channel,).
In case of multiple occurrences of the maximum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
argmax_channel(x) = [ 2., 2.]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L97
data | The input array |
|
inline |
Returns indices of the minimum values along an axis.
In the case of multiple occurrences of minimum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
// argmin along axis 0 argmin(x, axis=0) = [ 0., 0., 0.]
// argmin along axis 1 argmin(x, axis=1) = [ 0., 0.]
// argmin along axis 1 keeping same dims as an input array argmin(x, axis=1, keepdims=True) = [[ 0.], [ 0.]]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L77
symbol_name | name of the resulting symbol |
data | The input |
axis | The axis along which to perform the reduction. Negative values means indexing from right to left. `Requires axis to be set as int, because global \param keepdims If this is set to True`, the reduced axis is left in the result as |
|
inline |
Returns indices of the minimum values along an axis.
In the case of multiple occurrences of minimum values, the indices are returned.
Examples::
x = [[ 0., 1., 2.], [ 3., 4., 5.]]
// argmin along axis 0 argmin(x, axis=0) = [ 0., 0., 0.]
// argmin along axis 1 argmin(x, axis=1) = [ 0., 0.]
// argmin along axis 1 keeping same dims as an input array argmin(x, axis=1, keepdims=True) = [[ 0.], [ 0.]]
Defined in src/operator/tensor/broadcast_reduce_op_index.cc:L77
data | The input |
axis | The axis along which to perform the reduction. Negative values means indexing from right to left. `Requires axis to be set as int, because global \param keepdims If this is set to True`, the reduced axis is left in the result as |
|
inline |
Returns the indices that would sort an input array along the given axis.
This function performs sorting along the given axis and returns an array of as an input array that index data in sorted order.
Examples::
x = [[ 0.3, 0.2, 0.4], [ 0.1, 0.3, 0.2]]
// sort along axis -1 argsort(x) = [[ 1., 0., 2.], [ 0., 2., 1.]]
// sort along axis 0 argsort(x, axis=0) = [[ 1., 0., 1.] [ 0., 1., 0.]]
// flatten and then sort argsort(x) = [ 3., 1., 5., 0., 4., 2.]
Defined in src/operator/tensor/ordering_op.cc:L177
symbol_name | name of the resulting symbol |
data | The input array |
axis | Axis along which to sort the input tensor. If not given, the flattened |
is_ascend | Whether to sort in ascending or descending order. |
dtype | DType of the output indices. It is only valid when ret_typ is "indices" or "both". An error will be raised if the selected data type cannot precisely |
|
inline |
Returns the indices that would sort an input array along the given axis.
This function performs sorting along the given axis and returns an array of as an input array that index data in sorted order.
Examples::
x = [[ 0.3, 0.2, 0.4], [ 0.1, 0.3, 0.2]]
// sort along axis -1 argsort(x) = [[ 1., 0., 2.], [ 0., 2., 1.]]
// sort along axis 0 argsort(x, axis=0) = [[ 1., 0., 1.] [ 0., 1., 0.]]
// flatten and then sort argsort(x) = [ 3., 1., 5., 0., 4., 2.]
Defined in src/operator/tensor/ordering_op.cc:L177
data | The input array |
axis | Axis along which to sort the input tensor. If not given, the flattened |
is_ascend | Whether to sort in ascending or descending order. |
dtype | DType of the output indices. It is only valid when ret_typ is "indices" or "both". An error will be raised if the selected data type cannot precisely |
|
inline |
Batchwise dot product.
batch_dot
is used to compute dot product of x
and y
when x
and y
are data in batch, namely 3D arrays in shape of (batch_size, :, :)
.
For example, given x
with shape (batch_size, n, m)
and y
with shape (batch_size, m, k)
, the result array will have shape (batch_size, n, k)
, which is computed by::
batch_dot(x,y)[i,:,:] = dot(x[i,:,:], y[i,:,:])
Defined in src/operator/tensor/dot.cc:L125
symbol_name | name of the resulting symbol |
lhs | The first input |
rhs | The second input |
transpose_a | If true then transpose the first input before dot. |
transpose_b | If true then transpose the second input before dot. |
forward_stype | The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still |
|
inline |
Batchwise dot product.
batch_dot
is used to compute dot product of x
and y
when x
and y
are data in batch, namely 3D arrays in shape of (batch_size, :, :)
.
For example, given x
with shape (batch_size, n, m)
and y
with shape (batch_size, m, k)
, the result array will have shape (batch_size, n, k)
, which is computed by::
batch_dot(x,y)[i,:,:] = dot(x[i,:,:], y[i,:,:])
Defined in src/operator/tensor/dot.cc:L125
lhs | The first input |
rhs | The second input |
transpose_a | If true then transpose the first input before dot. |
transpose_b | If true then transpose the second input before dot. |
forward_stype | The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still |
Takes elements from a data batch.
.. note:: batch_take
is deprecated. Use pick
instead.
Given an input array of shape (d0, d1)
and indices of shape (i0,)
, the an output array of shape (i0,)
with::
output[i] = input[i, indices[i]]
Examples::
x = [[ 1., 2.], [ 3., 4.], [ 5., 6.]]
// takes elements with specified indices batch_take(x, [0,1,0]) = [ 1. 4. 5.]
Defined in src/operator/tensor/indexing_op.cc:L753
symbol_name | name of the resulting symbol |
a | The input array |
indices | The index array |
Takes elements from a data batch.
.. note:: batch_take
is deprecated. Use pick
instead.
Given an input array of shape (d0, d1)
and indices of shape (i0,)
, the an output array of shape (i0,)
with::
output[i] = input[i, indices[i]]
Examples::
x = [[ 1., 2.], [ 3., 4.], [ 5., 6.]]
// takes elements with specified indices batch_take(x, [0,1,0]) = [ 1. 4. 5.]
Defined in src/operator/tensor/indexing_op.cc:L753
a | The input array |
indices | The index array |
|
inline |
Batch normalization.
Normalizes a data batch by mean and variance, and applies a scale gamma
as well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:
.. math::
data_mean[i] = mean(data[:,i,:,...]) \ data_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as
.. math::
out[:,i,:,...] = {data[:,i,:,...] -
Both mean and var returns a scalar by treating the input as a vector.
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both the inverse of data_var
, which are needed for the backward pass. Note that two outputs are blocked.
Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean
and moving_var
, which are k-length vectors. They are global statistics for the whole dataset, which are updated by::
moving_mean = moving_mean * momentum + data_mean * (1 - momentum) moving_var = moving_var * momentum + data_var * (1 - momentum)
If use_global_stats
is set to be true, then moving_mean
and moving_var
are used instead of data_mean
and data_var
to compute the output. It is often used during inference.
The parameter axis
specifies which axis of the input shape denotes the 'channel' (separately normalized groups). The default is 1. Specifying -1 axis to be the last item in the input shape.
Both gamma
and beta
are learnable parameters. But if fix_gamma
is then set gamma
to 1 and its gradient to 0.
.. Note:: When fix_gamma
is set to True, no sparse support is provided. If the sparse tensors will fallback.
Defined in src/operator/nn/batch_norm.cc:L572
symbol_name | name of the resulting symbol |
data | Input data to batch normalization |
gamma | gamma array |
beta | beta array |
moving_mean | running mean of input |
moving_var | running variance of input |
eps | Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined |
momentum | Momentum for moving average |
fix_gamma | Fix gamma while training |
use_global_stats | Whether use global moving statistics instead of local |
output_mean_var | Output the mean and inverse std |
axis | Specify which shape axis the channel is specified |
cudnn_off | Do not select CUDNN operator, if available |
|
inline |
Batch normalization.
Normalizes a data batch by mean and variance, and applies a scale gamma
as well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:
.. math::
data_mean[i] = mean(data[:,i,:,...]) \ data_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as
.. math::
out[:,i,:,...] = {data[:,i,:,...] -
Both mean and var returns a scalar by treating the input as a vector.
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both the inverse of data_var
, which are needed for the backward pass. Note that two outputs are blocked.
Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean
and moving_var
, which are k-length vectors. They are global statistics for the whole dataset, which are updated by::
moving_mean = moving_mean * momentum + data_mean * (1 - momentum) moving_var = moving_var * momentum + data_var * (1 - momentum)
If use_global_stats
is set to be true, then moving_mean
and moving_var
are used instead of data_mean
and data_var
to compute the output. It is often used during inference.
The parameter axis
specifies which axis of the input shape denotes the 'channel' (separately normalized groups). The default is 1. Specifying -1 axis to be the last item in the input shape.
Both gamma
and beta
are learnable parameters. But if fix_gamma
is then set gamma
to 1 and its gradient to 0.
.. Note:: When fix_gamma
is set to True, no sparse support is provided. If the sparse tensors will fallback.
Defined in src/operator/nn/batch_norm.cc:L572
data | Input data to batch normalization |
gamma | gamma array |
beta | beta array |
moving_mean | running mean of input |
moving_var | running variance of input |
eps | Epsilon to prevent div 0. Must be no less than CUDNN_BN_MIN_EPSILON defined |
momentum | Momentum for moving average |
fix_gamma | Fix gamma while training |
use_global_stats | Whether use global moving statistics instead of local |
output_mean_var | Output the mean and inverse std |
axis | Specify which shape axis the channel is specified |
cudnn_off | Do not select CUDNN operator, if available |
|
inline |
Batch normalization.
This operator is DEPRECATED. Perform BatchNorm on the input.
Normalizes a data batch by mean and variance, and applies a scale gamma
as well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:
.. math::
data_mean[i] = mean(data[:,i,:,...]) \ data_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as
.. math::
out[:,i,:,...] = {data[:,i,:,...] -
Both mean and var returns a scalar by treating the input as a vector.
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both data_var
as well, which are needed for the backward pass.
Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean
and moving_var
, which are k-length vectors. They are global statistics for the whole dataset, which are updated by::
moving_mean = moving_mean * momentum + data_mean * (1 - momentum) moving_var = moving_var * momentum + data_var * (1 - momentum)
If use_global_stats
is set to be true, then moving_mean
and moving_var
are used instead of data_mean
and data_var
to compute the output. It is often used during inference.
Both gamma
and beta
are learnable parameters. But if fix_gamma
is then set gamma
to 1 and its gradient to 0.
There's no sparse support for this operator, and it will exhibit problematic sparse tensors.
Defined in src/operator/batch_norm_v1.cc:L95
symbol_name | name of the resulting symbol |
data | Input data to batch normalization |
gamma | gamma array |
beta | beta array |
eps | Epsilon to prevent div 0 |
momentum | Momentum for moving average |
fix_gamma | Fix gamma while training |
use_global_stats | Whether use global moving statistics instead of local |
output_mean_var | Output All,normal mean and var |
|
inline |
Batch normalization.
This operator is DEPRECATED. Perform BatchNorm on the input.
Normalizes a data batch by mean and variance, and applies a scale gamma
as well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis:
.. math::
data_mean[i] = mean(data[:,i,:,...]) \ data_var[i] = var(data[:,i,:,...])
Then compute the normalized output, which has the same shape as input, as
.. math::
out[:,i,:,...] = {data[:,i,:,...] -
Both mean and var returns a scalar by treating the input as a vector.
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both data_var
as well, which are needed for the backward pass.
Besides the inputs and the outputs, this operator accepts two auxiliary states, moving_mean
and moving_var
, which are k-length vectors. They are global statistics for the whole dataset, which are updated by::
moving_mean = moving_mean * momentum + data_mean * (1 - momentum) moving_var = moving_var * momentum + data_var * (1 - momentum)
If use_global_stats
is set to be true, then moving_mean
and moving_var
are used instead of data_mean
and data_var
to compute the output. It is often used during inference.
Both gamma
and beta
are learnable parameters. But if fix_gamma
is then set gamma
to 1 and its gradient to 0.
There's no sparse support for this operator, and it will exhibit problematic sparse tensors.
Defined in src/operator/batch_norm_v1.cc:L95
data | Input data to batch normalization |
gamma | gamma array |
beta | beta array |
eps | Epsilon to prevent div 0 |
momentum | Momentum for moving average |
fix_gamma | Fix gamma while training |
use_global_stats | Whether use global moving statistics instead of local |
output_mean_var | Output All,normal mean and var |
|
inline |
Applies bilinear sampling to input feature map.
Bilinear Sampling is the key of [NIPS2015] "Spatial Transformer Networks". except that the operator has the backward pass.
Given :math:data
and :math:grid
, then the output is computed by
.. math:: x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \ y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \ output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src},
:math:x_{dst}
, :math:y_{dst}
enumerate all spatial locations in The out-boundary points will be padded with zeros.The shape of the output will
The operator assumes that :math:data
has 'NCHW' layout and :math:grid
has
BilinearSampler often cooperates with GridGenerator which generates sampling GridGenerator supports two kinds of transformation: affine
and warp
. If users want to design a CustomOp to manipulate :math:grid
, please firstly
Example 1::
data = array([[[[1, 4, 3, 6], [1, 8, 8, 9], [0, 4, 1, 5], [1, 0, 1, 3]]]])
affine_matrix = array([[2, 0, 0], [0, 2, 0]])
affine_matrix = reshape(affine_matrix, shape=(1, 6))
grid = GridGenerator(data=affine_matrix, transform_type='affine',
out = BilinearSampler(data, grid)
out [[[[ 0, 0, 0, 0], [ 0, 3.5, 6.5, 0], [ 0, 1.25, 2.5, 0], [ 0, 0, 0, 0]]]
Example 2:: ## shift data horizontally by -1 pixel data = array([[[[1, 4, 3, 6], [1, 8, 8, 9], [0, 4, 1, 5], [1, 0, 1, 3]]]]) warp_maxtrix = array([[[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]], [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]]]) grid = GridGenerator(data=warp_matrix, transform_type='warp') out = BilinearSampler(data, grid) out [[[[ 4, 3, 6, 0], [ 8, 8, 9, 0], [ 4, 1, 5, 0], [ 0, 1, 3, 0]]] Defined in src/operator/bilinear_sampler.cc:L256
symbol_name | name of the resulting symbol |
data | Input data to the BilinearsamplerOp. |
grid | Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src |
cudnn_off | whether to turn cudnn off |
|
inline |
Applies bilinear sampling to input feature map.
Bilinear Sampling is the key of [NIPS2015] "Spatial Transformer Networks". except that the operator has the backward pass.
Given :math:data
and :math:grid
, then the output is computed by
.. math:: x_{src} = grid[batch, 0, y_{dst}, x_{dst}] \ y_{src} = grid[batch, 1, y_{dst}, x_{dst}] \ output[batch, channel, y_{dst}, x_{dst}] = G(data[batch, channel, y_{src},
:math:x_{dst}
, :math:y_{dst}
enumerate all spatial locations in The out-boundary points will be padded with zeros.The shape of the output will
The operator assumes that :math:data
has 'NCHW' layout and :math:grid
has
BilinearSampler often cooperates with GridGenerator which generates sampling GridGenerator supports two kinds of transformation: affine
and warp
. If users want to design a CustomOp to manipulate :math:grid
, please firstly
Example 1::
data = array([[[[1, 4, 3, 6], [1, 8, 8, 9], [0, 4, 1, 5], [1, 0, 1, 3]]]])
affine_matrix = array([[2, 0, 0], [0, 2, 0]])
affine_matrix = reshape(affine_matrix, shape=(1, 6))
grid = GridGenerator(data=affine_matrix, transform_type='affine',
out = BilinearSampler(data, grid)
out [[[[ 0, 0, 0, 0], [ 0, 3.5, 6.5, 0], [ 0, 1.25, 2.5, 0], [ 0, 0, 0, 0]]]
Example 2:: ## shift data horizontally by -1 pixel data = array([[[[1, 4, 3, 6], [1, 8, 8, 9], [0, 4, 1, 5], [1, 0, 1, 3]]]]) warp_maxtrix = array([[[[1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1]], [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]]]) grid = GridGenerator(data=warp_matrix, transform_type='warp') out = BilinearSampler(data, grid) out [[[[ 4, 3, 6, 0], [ 8, 8, 9, 0], [ 4, 1, 5, 0], [ 0, 1, 3, 0]]] Defined in src/operator/bilinear_sampler.cc:L256
data | Input data to the BilinearsamplerOp. |
grid | Input grid to the BilinearsamplerOp.grid has two channels: x_src, y_src |
cudnn_off | whether to turn cudnn off |
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator in the backward direction. In other words, this operator prevents the of its inputs to be taken into account for computing gradients.
Example::
v1 = [1, 2] v2 = [0, 1] a = Variable('a') b = Variable('b') b_stop_grad = stop_gradient(3 * b) loss = MakeLoss(b_stop_grad + a)
executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2)) executor.forward(is_train=True, a=v1, b=v2) executor.outputs [ 1. 5.]
executor.backward() executor.grad_arrays [ 0. 0.] [ 1. 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L299
symbol_name | name of the resulting symbol |
data | The input array. |
Stops gradient computation.
Stops the accumulated gradient of the inputs from flowing through this operator in the backward direction. In other words, this operator prevents the of its inputs to be taken into account for computing gradients.
Example::
v1 = [1, 2] v2 = [0, 1] a = Variable('a') b = Variable('b') b_stop_grad = stop_gradient(3 * b) loss = MakeLoss(b_stop_grad + a)
executor = loss.simple_bind(ctx=cpu(), a=(1,2), b=(1,2)) executor.forward(is_train=True, a=v1, b=v2) executor.outputs [ 1. 5.]
executor.backward() executor.grad_arrays [ 0. 0.] [ 1. 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L299
data | The input array. |
Returns element-wise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_add(x, y) = [[ 1., 1., 1.], [ 2., 2., 2.]]
broadcast_plus(x, y) = [[ 1., 1., 1.], [ 2., 2., 2.]]
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise sum of the input arrays with broadcasting.
broadcast_plus
is an alias to the function broadcast_add
.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_add(x, y) = [[ 1., 1., 1.], [ 2., 2., 2.]]
broadcast_plus(x, y) = [[ 1., 1., 1.], [ 2., 2., 2.]]
Supported sparse operations:
broadcast_add(csr, dense(1D)) = dense broadcast_add(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L58
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
Example::
// given x of shape (1,2,1) x = [[[ 1.], [ 2.]]]
// broadcast x on on axis 2 broadcast_axis(x, axis=2, size=3) = [[[ 1., 1., 1.], [ 2., 2., 2.]]] // broadcast x on on axes 0 and 2 broadcast_axis(x, axis=(0,2), size=(2,3)) = [[[ 1., 1., 1.], [ 2., 2., 2.]], [[ 1., 1., 1.], [ 2., 2., 2.]]]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L238
symbol_name | name of the resulting symbol |
data | The input |
axis | The axes to perform the broadcasting. |
size | Target sizes of the broadcasting axes. |
|
inline |
Broadcasts the input array over particular axes.
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
Example::
// given x of shape (1,2,1) x = [[[ 1.], [ 2.]]]
// broadcast x on on axis 2 broadcast_axis(x, axis=2, size=3) = [[[ 1., 1., 1.], [ 2., 2., 2.]]] // broadcast x on on axes 0 and 2 broadcast_axis(x, axis=(0,2), size=(2,3)) = [[[ 1., 1., 1.], [ 2., 2., 2.]], [[ 1., 1., 1.], [ 2., 2., 2.]]]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L238
data | The input |
axis | The axes to perform the broadcasting. |
size | Target sizes of the broadcasting axes. |
Returns element-wise division of the input arrays with broadcasting.
Example::
x = [[ 6., 6., 6.], [ 6., 6., 6.]]
y = [[ 2.], [ 3.]]
broadcast_div(x, y) = [[ 3., 3., 3.], [ 2., 2., 2.]]
Supported sparse operations:
broadcast_div(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L187
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise division of the input arrays with broadcasting.
Example::
x = [[ 6., 6., 6.], [ 6., 6., 6.]]
y = [[ 2.], [ 3.]]
broadcast_div(x, y) = [[ 3., 3., 3.], [ 2., 2., 2.]]
Supported sparse operations:
broadcast_div(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L187
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise equal to (==) comparison operation with.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_equal(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L46
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise equal to (==) comparison operation with.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_equal(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L46
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise greater than (>) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_greater(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L82
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise greater than (>) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_greater(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L82
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise greater than or equal to (>=) comparison.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_greater_equal(x, y) = [[ 1., 1., 1.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L100
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise greater than or equal to (>=) comparison.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_greater_equal(x, y) = [[ 1., 1., 1.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L100
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the hypotenuse of a right angled triangle, given its "legs" with broadcasting.
It is equivalent to doing :math:sqrt(x_1^2 + x_2^2)
.
Example::
x = [[ 3., 3., 3.]]
y = [[ 4.], [ 4.]]
broadcast_hypot(x, y) = [[ 5., 5., 5.], [ 5., 5., 5.]]
z = [[ 0.], [ 4.]]
broadcast_hypot(x, z) = [[ 3., 3., 3.], [ 5., 5., 5.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L156
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the hypotenuse of a right angled triangle, given its "legs" with broadcasting.
It is equivalent to doing :math:sqrt(x_1^2 + x_2^2)
.
Example::
x = [[ 3., 3., 3.]]
y = [[ 4.], [ 4.]]
broadcast_hypot(x, y) = [[ 5., 5., 5.], [ 5., 5., 5.]]
z = [[ 0.], [ 4.]]
broadcast_hypot(x, z) = [[ 3., 3., 3.], [ 5., 5., 5.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L156
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise lesser than (<) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_lesser(x, y) = [[ 0., 0., 0.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L118
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise lesser than (<) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_lesser(x, y) = [[ 0., 0., 0.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L118
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise lesser than or equal to (<=) comparison.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_lesser_equal(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L136
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise lesser than or equal to (<=) comparison.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_lesser_equal(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L136
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Broadcasts lhs to have the same shape as rhs.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic with arrays of different shapes efficiently without creating multiple copies of Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_like([[1,2,3]], [[5,6,7],[7,8,9]]) = [[ 1., 2., 3.], [ 1., 2., 3.]])
broadcast_like([9], [1,2,3,4,5], lhs_axes=(0,), rhs_axes=(-1,)) = [9,9,9,9,9]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L315
symbol_name | name of the resulting symbol |
lhs | First input. |
rhs | Second input. |
lhs_axes | Axes to perform broadcast on in the first input array |
rhs_axes | Axes to copy from the second input array |
|
inline |
Broadcasts lhs to have the same shape as rhs.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic with arrays of different shapes efficiently without creating multiple copies of Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_like([[1,2,3]], [[5,6,7],[7,8,9]]) = [[ 1., 2., 3.], [ 1., 2., 3.]])
broadcast_like([9], [1,2,3,4,5], lhs_axes=(0,), rhs_axes=(-1,)) = [9,9,9,9,9]
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L315
lhs | First input. |
rhs | Second input. |
lhs_axes | Axes to perform broadcast on in the first input array |
rhs_axes | Axes to copy from the second input array |
|
inline |
Returns the result of element-wise logical and with broadcasting.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_logical_and(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L154
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise logical and with broadcasting.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_logical_and(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L154
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise logical or with broadcasting.
Example::
x = [[ 1., 1., 0.], [ 1., 1., 0.]]
y = [[ 1.], [ 0.]]
broadcast_logical_or(x, y) = [[ 1., 1., 1.], [ 1., 1., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L172
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise logical or with broadcasting.
Example::
x = [[ 1., 1., 0.], [ 1., 1., 0.]]
y = [[ 1.], [ 0.]]
broadcast_logical_or(x, y) = [[ 1., 1., 1.], [ 1., 1., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L172
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise logical xor with broadcasting.
Example::
x = [[ 1., 1., 0.], [ 1., 1., 0.]]
y = [[ 1.], [ 0.]]
broadcast_logical_xor(x, y) = [[ 0., 0., 1.], [ 1., 1., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L190
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise logical xor with broadcasting.
Example::
x = [[ 1., 1., 0.], [ 1., 1., 0.]]
y = [[ 1.], [ 0.]]
broadcast_logical_xor(x, y) = [[ 0., 0., 1.], [ 1., 1., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L190
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns element-wise maximum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_maximum(x, y) = [[ 1., 1., 1.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L80
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise maximum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_maximum(x, y) = [[ 1., 1., 1.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L80
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns element-wise minimum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_maximum(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L115
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise minimum of the input arrays with broadcasting.
This function compares two input arrays and returns a new array having the
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_maximum(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L115
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise modulo of the input arrays with broadcasting.
Example::
x = [[ 8., 8., 8.], [ 8., 8., 8.]]
y = [[ 2.], [ 3.]]
broadcast_mod(x, y) = [[ 0., 0., 0.], [ 2., 2., 2.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L222
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise modulo of the input arrays with broadcasting.
Example::
x = [[ 8., 8., 8.], [ 8., 8., 8.]]
y = [[ 2.], [ 3.]]
broadcast_mod(x, y) = [[ 0., 0., 0.], [ 2., 2., 2.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L222
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise product of the input arrays with broadcasting.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_mul(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Supported sparse operations:
broadcast_mul(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L146
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise product of the input arrays with broadcasting.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_mul(x, y) = [[ 0., 0., 0.], [ 1., 1., 1.]]
Supported sparse operations:
broadcast_mul(csr, dense(1D)) = csr
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L146
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns the result of element-wise not equal to (!=) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_not_equal(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L64
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns the result of element-wise not equal to (!=) comparison operation.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_not_equal(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_logic.cc:L64
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Returns result of first array elements raised to powers from second array,.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_power(x, y) = [[ 2., 2., 2.], [ 4., 4., 4.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L45
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns result of first array elements raised to powers from second array,.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_power(x, y) = [[ 2., 2., 2.], [ 4., 4., 4.]]
Defined in src/operator/tensor/elemwise_binary_broadcast_op_extended.cc:L45
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_sub(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
broadcast_minus(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
symbol_name | name of the resulting symbol |
lhs | First input to the function |
rhs | Second input to the function |
Returns element-wise difference of the input arrays with broadcasting.
broadcast_minus
is an alias to the function broadcast_sub
.
Example::
x = [[ 1., 1., 1.], [ 1., 1., 1.]]
y = [[ 0.], [ 1.]]
broadcast_sub(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
broadcast_minus(x, y) = [[ 1., 1., 1.], [ 0., 0., 0.]]
Supported sparse operations:
broadcast_sub/minus(csr, dense(1D)) = dense broadcast_sub/minus(dense(1D), csr) = dense
Defined in src/operator/tensor/elemwise_binary_broadcast_op_basic.cc:L106
lhs | First input to the function |
rhs | Second input to the function |
|
inline |
Broadcasts the input array to a new shape.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic with arrays of different shapes efficiently without creating multiple copies of Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_to([[1,2,3]], shape=(2,3)) = [[ 1., 2., 3.], [ 1., 2., 3.]])
The dimension which you do not want to change can also be kept as 0
which So with shape=(2,0)
, we will obtain the same result as in the above example.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L262
symbol_name | name of the resulting symbol |
data | The input |
shape | The shape of the desired array. We can set the dim to zero if it's same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0)) has the same |
Broadcasts the input array to a new shape.
Broadcasting is a mechanism that allows NDArrays to perform arithmetic with arrays of different shapes efficiently without creating multiple copies of Also see, Broadcasting <https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html>
_ for more
Broadcasting is allowed on axes with size 1, such as from (2,1,3,1)
to (2,8,3,9)
. Elements will be duplicated on the broadcasted axes.
For example::
broadcast_to([[1,2,3]], shape=(2,3)) = [[ 1., 2., 3.], [ 1., 2., 3.]])
The dimension which you do not want to change can also be kept as 0
which So with shape=(2,0)
, we will obtain the same result as in the above example.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L262
data | The input |
shape | The shape of the desired array. We can set the dim to zero if it's same as the original. E.g A = broadcast_to(B, shape=(10, 0, 0)) has the same |
Casts all elements of the input to a new type.
.. note:: Cast
is deprecated. Use cast
instead.
Example::
cast([0.9, 1.3], dtype='int32') = [0, 1] cast([1e20, 11.1], dtype='float16') = [inf, 11.09375] cast([300, 11.1, 10.9, -1, -3], dtype='uint8') = [44, 11, 10, 255, 253]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L634
symbol_name | name of the resulting symbol |
data | The input. |
dtype | Output data type. |
Casts all elements of the input to a new type.
.. note:: Cast
is deprecated. Use cast
instead.
Example::
cast([0.9, 1.3], dtype='int32') = [0, 1] cast([1e20, 11.1], dtype='float16') = [inf, 11.09375] cast([300, 11.1, 10.9, -1, -3], dtype='uint8') = [44, 11, 10, 255, 253]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L634
data | The input. |
dtype | Output data type. |
|
inline |
Casts tensor storage type to the new type.
When an NDArray with default storage type is cast to csr or row_sparse storage, the result is compact, which means:
The storage type of cast_storage
output depends on stype parameter:
Example::
dense = [[ 0., 1., 0.], [ 2., 0., 3.], [ 0., 0., 0.], [ 0., 0., 0.]]
rsp = cast_storage(dense, 'row_sparse') rsp.indices = [0, 1] rsp.values = [[ 0., 1., 0.], [ 2., 0., 3.]]
csr = cast_storage(dense, 'csr') csr.indices = [1, 0, 2] csr.values = [ 1., 2., 3.] csr.indptr = [0, 1, 3, 3, 3]
Defined in src/operator/tensor/cast_storage.cc:L71
symbol_name | name of the resulting symbol |
data | The input. |
stype | Output storage type. |
|
inline |
Casts tensor storage type to the new type.
When an NDArray with default storage type is cast to csr or row_sparse storage, the result is compact, which means:
The storage type of cast_storage
output depends on stype parameter:
Example::
dense = [[ 0., 1., 0.], [ 2., 0., 3.], [ 0., 0., 0.], [ 0., 0., 0.]]
rsp = cast_storage(dense, 'row_sparse') rsp.indices = [0, 1] rsp.values = [[ 0., 1., 0.], [ 2., 0., 3.]]
csr = cast_storage(dense, 'csr') csr.indices = [1, 0, 2] csr.values = [ 1., 2., 3.] csr.indptr = [0, 1, 3, 3, 3]
Defined in src/operator/tensor/cast_storage.cc:L71
data | The input. |
stype | Output storage type. |
Returns element-wise cube-root value of the input.
.. math:: cbrt(x) = [3]{x}
Example::
cbrt([1, 8, -125]) = [1, 2, -5]
The storage type of cbrt
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L950
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise cube-root value of the input.
.. math:: cbrt(x) = [3]{x}
Example::
cbrt([1, 8, -125]) = [1, 2, -5]
The storage type of cbrt
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L950
data | The input array. |
Returns element-wise ceiling of the input.
The ceil of the scalar x is the smallest integer i, such that i >= x.
Example::
ceil([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-2., -1., 2., 2., 3.]
The storage type of ceil
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L786
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise ceiling of the input.
The ceil of the scalar x is the smallest integer i, such that i >= x.
Example::
ceil([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-2., -1., 2., 2., 3.]
The storage type of ceil
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L786
data | The input array. |
|
inline |
Clips (limits) the values in an array.
Given an interval, values outside the interval are clipped to the interval Clipping x
between a_min
and a_x
would be::
clip(x, a_min, a_max) = max(min(x, a_max), a_min))
Example::
x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
clip(x,1,8) = [ 1., 1., 2., 3., 4., 5., 6., 7., 8., 8.]
The storage type of clip
output depends on storage types of inputs and the parameter values:
Defined in src/operator/tensor/matrix_op.cc:L723
symbol_name | name of the resulting symbol |
data | Input array. |
a_min | Minimum value |
a_max | Maximum value |
Clips (limits) the values in an array.
Given an interval, values outside the interval are clipped to the interval Clipping x
between a_min
and a_x
would be::
clip(x, a_min, a_max) = max(min(x, a_max), a_min))
Example::
x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
clip(x,1,8) = [ 1., 1., 2., 3., 4., 5., 6., 7., 8., 8.]
The storage type of clip
output depends on storage types of inputs and the parameter values:
Defined in src/operator/tensor/matrix_op.cc:L723
data | Input array. |
a_min | Minimum value |
a_max | Maximum value |
|
inline |
Joins input arrays along a given axis.
.. note:: Concat
is deprecated. Use concat
instead.
The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.
The storage type of concat
output depends on storage types of inputs
concat
generates output with default storageExample::
x = [[1,1],[2,2]] y = [[3,3],[4,4],[5,5]] z = [[6,6], [7,7],[8,8]]
concat(x,y,z,dim=0) = [[ 1., 1.], [ 2., 2.], [ 3., 3.], [ 4., 4.], [ 5., 5.], [ 6., 6.], [ 7., 7.], [ 8., 8.]]
Note that you cannot concat x,y,z along dimension 1 since dimension 0 is not the same for all the input arrays.
concat(y,z,dim=1) = [[ 3., 3., 6., 6.], [ 4., 4., 7., 7.], [ 5., 5., 8., 8.]]
Defined in src/operator/nn/concat.cc:L371
symbol_name | name of the resulting symbol |
data | List of arrays to concatenate |
num_args | Number of inputs to be concated. |
dim | the dimension to be concated. |
Joins input arrays along a given axis.
.. note:: Concat
is deprecated. Use concat
instead.
The dimensions of the input arrays should be the same except the axis along which they will be concatenated. The dimension of the output array along the concatenated axis will be equal to the sum of the corresponding dimensions of the input arrays.
The storage type of concat
output depends on storage types of inputs
concat
generates output with default storageExample::
x = [[1,1],[2,2]] y = [[3,3],[4,4],[5,5]] z = [[6,6], [7,7],[8,8]]
concat(x,y,z,dim=0) = [[ 1., 1.], [ 2., 2.], [ 3., 3.], [ 4., 4.], [ 5., 5.], [ 6., 6.], [ 7., 7.], [ 8., 8.]]
Note that you cannot concat x,y,z along dimension 1 since dimension 0 is not the same for all the input arrays.
concat(y,z,dim=1) = [[ 3., 3., 6., 6.], [ 4., 4., 7., 7.], [ 5., 5., 8., 8.]]
Defined in src/operator/nn/concat.cc:L371
data | List of arrays to concatenate |
num_args | Number of inputs to be concated. |
dim | the dimension to be concated. |
|
inline |
Compute N-D convolution on *(N+2)*-D input.
In the 2-D convolution, given input data with shape *(batch_size, channel, height, width)*, the output is computed by
.. math::
out[n,i,:,:] = bias[i] + {j=0}^{channel} data[n,j,:,:] weight[i,j,:,:]
where :math:\star
is the 2-D cross-correlation operator.
For general 2-D convolution, the shapes are
Define::
f(x,k,p,s,d) = floor((x+2*p-d*(k-1)-1)/s)+1
then we have::
out_height=f(height, kernel[0], pad[0], stride[0], dilate[0]) out_width=f(width, kernel[1], pad[1], stride[1], dilate[1])
If no_bias
is set to be true, then the bias
term is ignored.
The default data layout
is NCHW, namely *(batch_size, channel, height, width)*. We can choose other layouts such as NWC.
If num_group
is larger than 1, denoted by g, then split the input data
evenly into g parts along the channel axis, and also evenly split weight
along the first dimension. Next compute the convolution on the i-th part of the data with the i-th weight part. The output is obtained by concatenating the g results.
1-D convolution does not have height dimension but only width in space.
3-D convolution adds an additional depth dimension besides height and width. The shapes are
Both weight
and bias
are learnable parameters.
There are other options to tune the performance.
MXNET_CUDNN_AUTOTUNE_DEFAULT
. 0 for off, 1 for limited workspace (default), 2 for fastest.Defined in src/operator/nn/convolution.cc:L472
symbol_name | name of the resulting symbol |
data | Input data to the ConvolutionOp. |
weight | Weight matrix. |
bias | Bias parameter. |
kernel | Convolution kernel size: (w,), (h, w) or (d, h, w) |
num_filter | Convolution filter(channel) number |
stride | Convolution stride: (w,), (h, w) or (d, h, w). Defaults to 1 for each |
dilate | Convolution dilate: (w,), (h, w) or (d, h, w). Defaults to 1 for each |
pad | Zero pad for convolution: (w,), (h, w) or (d, h, w). Defaults to no padding. |
num_group | Number of group partitions. |
workspace | Maximum temporary workspace allowed (MB) in convolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the convolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algo by running performance test. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d.NHWC and NDHWC are |
|
inline |
Compute N-D convolution on *(N+2)*-D input.
In the 2-D convolution, given input data with shape *(batch_size, channel, height, width)*, the output is computed by
.. math::
out[n,i,:,:] = bias[i] + {j=0}^{channel} data[n,j,:,:] weight[i,j,:,:]
where :math:\star
is the 2-D cross-correlation operator.
For general 2-D convolution, the shapes are
Define::
f(x,k,p,s,d) = floor((x+2*p-d*(k-1)-1)/s)+1
then we have::
out_height=f(height, kernel[0], pad[0], stride[0], dilate[0]) out_width=f(width, kernel[1], pad[1], stride[1], dilate[1])
If no_bias
is set to be true, then the bias
term is ignored.
The default data layout
is NCHW, namely *(batch_size, channel, height, width)*. We can choose other layouts such as NWC.
If num_group
is larger than 1, denoted by g, then split the input data
evenly into g parts along the channel axis, and also evenly split weight
along the first dimension. Next compute the convolution on the i-th part of the data with the i-th weight part. The output is obtained by concatenating the g results.
1-D convolution does not have height dimension but only width in space.
3-D convolution adds an additional depth dimension besides height and width. The shapes are
Both weight
and bias
are learnable parameters.
There are other options to tune the performance.
MXNET_CUDNN_AUTOTUNE_DEFAULT
. 0 for off, 1 for limited workspace (default), 2 for fastest.Defined in src/operator/nn/convolution.cc:L472
data | Input data to the ConvolutionOp. |
weight | Weight matrix. |
bias | Bias parameter. |
kernel | Convolution kernel size: (w,), (h, w) or (d, h, w) |
num_filter | Convolution filter(channel) number |
stride | Convolution stride: (w,), (h, w) or (d, h, w). Defaults to 1 for each |
dilate | Convolution dilate: (w,), (h, w) or (d, h, w). Defaults to 1 for each |
pad | Zero pad for convolution: (w,), (h, w) or (d, h, w). Defaults to no padding. |
num_group | Number of group partitions. |
workspace | Maximum temporary workspace allowed (MB) in convolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the convolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algo by running performance test. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout: NCW for 1d, NCHW for 2d and NCDHW for 3d.NHWC and NDHWC are |
|
inline |
This operator is DEPRECATED. Apply convolution to input then add a bias.
symbol_name | name of the resulting symbol |
data | Input data to the ConvolutionV1Op. |
weight | Weight matrix. |
bias | Bias parameter. |
kernel | convolution kernel size: (h, w) or (d, h, w) |
num_filter | convolution filter(channel) number |
stride | convolution stride: (h, w) or (d, h, w) |
dilate | convolution dilate: (h, w) or (d, h, w) |
pad | pad for convolution: (h, w) or (d, h, w) |
num_group | Number of group partitions. Equivalent to slicing input into num_group partitions, apply convolution on each, then concatenate the results |
workspace | Maximum temporary workspace allowed for convolution (MB).This parameter determines the effective batch size of the convolution kernel, which may be smaller than the given batch size. Also, the workspace will be |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: 'off': no tuning 'limited_workspace': run test and pick the fastest algorithm that doesn't 'fastest': pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d. |
|
inline |
This operator is DEPRECATED. Apply convolution to input then add a bias.
data | Input data to the ConvolutionV1Op. |
weight | Weight matrix. |
bias | Bias parameter. |
kernel | convolution kernel size: (h, w) or (d, h, w) |
num_filter | convolution filter(channel) number |
stride | convolution stride: (h, w) or (d, h, w) |
dilate | convolution dilate: (h, w) or (d, h, w) |
pad | pad for convolution: (h, w) or (d, h, w) |
num_group | Number of group partitions. Equivalent to slicing input into num_group partitions, apply convolution on each, then concatenate the results |
workspace | Maximum temporary workspace allowed for convolution (MB).This parameter determines the effective batch size of the convolution kernel, which may be smaller than the given batch size. Also, the workspace will be |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algo by running performance test. Leads to higher startup time but may give faster speed. Options are: 'off': no tuning 'limited_workspace': run test and pick the fastest algorithm that doesn't 'fastest': pick the fastest algorithm and ignore workspace limit. If set to None (default), behavior is determined by environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT: 0 for off, 1 for limited workspace (default), 2 for fastest. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout: NCHW for 2d and NCDHW for 3d. |
|
inline |
Applies correlation to inputs.
The correlation layer performs multiplicative patch comparisons between two
Given two multi-channel feature maps :math:f_{1}, f_{2}
, with :math:w
, the correlation layer lets the network compare each patch from :math:f_{1}
For now we consider only a single comparison of two patches. The 'correlation' :math:x_{2}
in the second map is then defined as:
.. math::
c(x_{1}, x_{2}) = {o [-k,k] [-k,k]} <f_{1}(x_{1} + o),
for a square patch of size :math:K:=2k+1
.
Note that the equation above is identical to one step of a convolution in neural networks, but instead of convolving data with a filter, it convolves data. For this reason, it has no training weights.
Computing :math:c(x_{1}, x_{2})
involves :math:c * K^{2}
multiplications.
Given a maximum displacement :math:d
, for each location :math:x_{1}
it computes correlations :math:c(x_{1}, x_{2})
only in a neighborhood of size by limiting the range of :math:x_{2}
. We use strides :math:s_{1}, s_{2}
, to quantize :math:x_{1}
globally and to quantize :math:x_{2}
within the centered around :math:x_{1}
.
The final output is defined by the following expression:
.. math:: out[n, q, i, j] = c(x_{i, j}, x_{q})
where :math:i
and :math:j
enumerate spatial locations in :math:f_{1}
, and
Defined in src/operator/correlation.cc:L198
symbol_name | name of the resulting symbol |
data1 | Input data1 to the correlation. |
data2 | Input data2 to the correlation. |
kernel_size | kernel size for Correlation must be an odd number |
max_displacement | Max displacement of Correlation |
stride1 | stride1 quantize data1 globally |
stride2 | stride2 quantize data2 within the neighborhood centered around data1 |
pad_size | pad for Correlation |
is_multiply | operation type is either multiplication or subduction |
|
inline |
Applies correlation to inputs.
The correlation layer performs multiplicative patch comparisons between two
Given two multi-channel feature maps :math:f_{1}, f_{2}
, with :math:w
, the correlation layer lets the network compare each patch from :math:f_{1}
For now we consider only a single comparison of two patches. The 'correlation' :math:x_{2}
in the second map is then defined as:
.. math::
c(x_{1}, x_{2}) = {o [-k,k] [-k,k]} <f_{1}(x_{1} + o),
for a square patch of size :math:K:=2k+1
.
Note that the equation above is identical to one step of a convolution in neural networks, but instead of convolving data with a filter, it convolves data. For this reason, it has no training weights.
Computing :math:c(x_{1}, x_{2})
involves :math:c * K^{2}
multiplications.
Given a maximum displacement :math:d
, for each location :math:x_{1}
it computes correlations :math:c(x_{1}, x_{2})
only in a neighborhood of size by limiting the range of :math:x_{2}
. We use strides :math:s_{1}, s_{2}
, to quantize :math:x_{1}
globally and to quantize :math:x_{2}
within the centered around :math:x_{1}
.
The final output is defined by the following expression:
.. math:: out[n, q, i, j] = c(x_{i, j}, x_{q})
where :math:i
and :math:j
enumerate spatial locations in :math:f_{1}
, and
Defined in src/operator/correlation.cc:L198
data1 | Input data1 to the correlation. |
data2 | Input data2 to the correlation. |
kernel_size | kernel size for Correlation must be an odd number |
max_displacement | Max displacement of Correlation |
stride1 | stride1 quantize data1 globally |
stride2 | stride2 quantize data2 within the neighborhood centered around data1 |
pad_size | pad for Correlation |
is_multiply | operation type is either multiplication or subduction |
Computes the element-wise cosine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
.. math:: cos([0, /4, /2]) = [1, 0.707, 0]
The storage type of cos
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L89
symbol_name | name of the resulting symbol |
data | The input array. |
Computes the element-wise cosine of the input array.
The input should be in radians (:math:2\pi
rad equals 360 degrees).
.. math:: cos([0, /4, /2]) = [1, 0.707, 0]
The storage type of cos
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L89
data | The input array. |
Returns the hyperbolic cosine of the input array, computed element-wise.
.. math:: cosh(x) = 0.5(exp(x) + exp(-x))
The storage type of cosh
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L272
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the hyperbolic cosine of the input array, computed element-wise.
.. math:: cosh(x) = 0.5(exp(x) + exp(-x))
The storage type of cosh
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L272
data | The input array. |
|
inline |
|
inline |
.. note:: Crop
is deprecated. Use slice
instead.
Crop the 2nd and 3rd dim of input data, with the corresponding size of h_w or with width and height of the second input symbol, i.e., with one input, we need specify the crop height and width, otherwise the second input symbol's size
Defined in src/operator/crop.cc:L50
symbol_name | name of the resulting symbol |
data | Tensor or List of Tensors, the second input will be used as crop_like |
num_args | Number of inputs for crop, if equals one, then we will use the h_wfor crop height and width, else if equals two, then we will use the heightand width |
offset | crop offset coordinate: (y, x) |
h_w | crop height and width: (h, w) |
center_crop | If set to true, then it will use be the center_crop,or it will crop |
|
inline |
.. note:: Crop
is deprecated. Use slice
instead.
Crop the 2nd and 3rd dim of input data, with the corresponding size of h_w or with width and height of the second input symbol, i.e., with one input, we need specify the crop height and width, otherwise the second input symbol's size
Defined in src/operator/crop.cc:L50
data | Tensor or List of Tensors, the second input will be used as crop_like |
num_args | Number of inputs for crop, if equals one, then we will use the h_wfor crop height and width, else if equals two, then we will use the heightand width |
offset | crop offset coordinate: (y, x) |
h_w | crop height and width: (h, w) |
center_crop | If set to true, then it will use be the center_crop,or it will crop |
|
inline |
Connectionist Temporal Classification Loss.
.. note:: The existing alias contrib_CTCLoss
is deprecated.
The shapes of the inputs and outputs:
(sequence_length, batch_size, alphabet_size)
(batch_size, label_sequence_length)
(batch_size)
The data
tensor consists of sequences of activation vectors (without applying with i-th channel in the last dimension corresponding to i-th label for i between 0 and alphabet_size-1 (i.e always 0-indexed). Alphabet size should include one additional value reserved for blank label. When blank_label
is "first"
, the 0
-th channel is be reserved for activation of blank label, or otherwise if it is "last", reserved for blank label.
label
is an index matrix of integers. When blank_label
is "first"
, the value 0 is then reserved for blank label, and should not be passed in this when blank_label
is "last"
, the value (alphabet_size-1)
is reserved for
If a sequence of labels is shorter than label_sequence_length, use the special padding value at the end of the sequence to conform it to the correct length. The padding value is 0
when blank_label
is "first"
, and -1
For example, suppose the vocabulary is [a, b, c]
, and in one batch we have 'ba', 'cbb', and 'abac'. When blank_label
is "first"
, we can index the `{'a': 1, 'b': 2, 'c': 3}, and we reserve the 0-th channel for blank label in The resulting
label` tensor should be padded to be::
[[2, 1, 0, 0], [3, 2, 2, 0], [1, 2, 1, 3]]
When blank_label
is "last"
, we can index the labels as `{'a': 0, 'b': 1, 'c': 2}, and we reserve the channel index 3 for blank label The resulting
label` tensor should be padded to be::
[[1, 0, -1, -1], [2, 1, 1, -1], [0, 1, 0, 2]]
out
is a list of CTC loss values, one per example in the batch.
See Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, A. Graves et al. for more information on the definition and the algorithm.
Defined in src/operator/nn/ctc_loss.cc:L100
symbol_name | name of the resulting symbol |
data | Input ndarray |
label | Ground-truth labels for the loss. |
data_lengths | Lengths of data for each of the samples. Only required when |
label_lengths | Lengths of labels for each of the samples. Only required when |
use_data_lengths | Whether the data lenghts are decided by data_lengths . If |
use_label_lengths | Whether the label lenghts are decided by label_lengths , or derived from padding_mask . If false, the lengths are derived from the first occurrence of the value of padding_mask . The value of padding_mask is 0 when first CTC label is reserved for blank, and -1 when last label is |
blank_label | Set the label that is reserved for blank label.If "first", 0-th label is reserved, and label values for tokens in the vocabulary are between 1 and alphabet_size-1 , and the padding mask is -1 . If "last", last label value alphabet_size-1 is reserved for blank label instead, and label values for tokens in the vocabulary are between 0 and alphabet_size-2 , |
|
inline |
Connectionist Temporal Classification Loss.
.. note:: The existing alias contrib_CTCLoss
is deprecated.
The shapes of the inputs and outputs:
(sequence_length, batch_size, alphabet_size)
(batch_size, label_sequence_length)
(batch_size)
The data
tensor consists of sequences of activation vectors (without applying with i-th channel in the last dimension corresponding to i-th label for i between 0 and alphabet_size-1 (i.e always 0-indexed). Alphabet size should include one additional value reserved for blank label. When blank_label
is "first"
, the 0
-th channel is be reserved for activation of blank label, or otherwise if it is "last", reserved for blank label.
label
is an index matrix of integers. When blank_label
is "first"
, the value 0 is then reserved for blank label, and should not be passed in this when blank_label
is "last"
, the value (alphabet_size-1)
is reserved for
If a sequence of labels is shorter than label_sequence_length, use the special padding value at the end of the sequence to conform it to the correct length. The padding value is 0
when blank_label
is "first"
, and -1
For example, suppose the vocabulary is [a, b, c]
, and in one batch we have 'ba', 'cbb', and 'abac'. When blank_label
is "first"
, we can index the `{'a': 1, 'b': 2, 'c': 3}, and we reserve the 0-th channel for blank label in The resulting
label` tensor should be padded to be::
[[2, 1, 0, 0], [3, 2, 2, 0], [1, 2, 1, 3]]
When blank_label
is "last"
, we can index the labels as `{'a': 0, 'b': 1, 'c': 2}, and we reserve the channel index 3 for blank label The resulting
label` tensor should be padded to be::
[[1, 0, -1, -1], [2, 1, 1, -1], [0, 1, 0, 2]]
out
is a list of CTC loss values, one per example in the batch.
See Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, A. Graves et al. for more information on the definition and the algorithm.
Defined in src/operator/nn/ctc_loss.cc:L100
data | Input ndarray |
label | Ground-truth labels for the loss. |
data_lengths | Lengths of data for each of the samples. Only required when |
label_lengths | Lengths of labels for each of the samples. Only required when |
use_data_lengths | Whether the data lenghts are decided by data_lengths . If |
use_label_lengths | Whether the label lenghts are decided by label_lengths , or derived from padding_mask . If false, the lengths are derived from the first occurrence of the value of padding_mask . The value of padding_mask is 0 when first CTC label is reserved for blank, and -1 when last label is |
blank_label | Set the label that is reserved for blank label.If "first", 0-th label is reserved, and label values for tokens in the vocabulary are between 1 and alphabet_size-1 , and the padding mask is -1 . If "last", last label value alphabet_size-1 is reserved for blank label instead, and label values for tokens in the vocabulary are between 0 and alphabet_size-2 , |
|
inline |
Apply a custom operator implemented in a frontend language (like Python).
Custom operators should override required methods like forward
and backward
. The custom operator must be registered before it can be used. Please check the tutorial here: /faq/new_op.html.
Defined in src/operator/custom/custom.cc:L546
symbol_name | name of the resulting symbol |
data | Input data for the custom operator. |
op_type | Name of the custom operator. This is the name that is passed to |
|
inline |
Apply a custom operator implemented in a frontend language (like Python).
Custom operators should override required methods like forward
and backward
. The custom operator must be registered before it can be used. Please check the tutorial here: /faq/new_op.html.
Defined in src/operator/custom/custom.cc:L546
data | Input data for the custom operator. |
op_type | Name of the custom operator. This is the name that is passed to |
|
inline |
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller.
symbol_name | name of the resulting symbol |
data | Input tensor to the deconvolution operation. |
weight | Weights representing the kernel. |
bias | Bias added to the result after the deconvolution operation. |
kernel | Deconvolution kernel size: (w,), (h, w) or (d, h, w). This is same as |
num_filter | Number of output filters. |
stride | The stride used for the corresponding convolution: (w,), (h, w) or (d, |
dilate | Dilation factor for each dimension of the input: (w,), (h, w) or (d, h, |
pad | The amount of implicit zero padding added during convolution for each dimension of the input: (w,), (h, w) or (d, h, w). (kernel-1)/2 is usually a good choice. If target_shape is set, pad will be ignored and a padding |
adj | Adjustment for output shape: (w,), (h, w) or (d, h, w). If target_shape |
target_shape | Shape of the output tensor: (w,), (h, w) or (d, h, w). |
num_group | Number of groups partition. |
workspace | Maximum temporary workspace allowed (MB) in deconvolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the deconvolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algorithm by running performance test. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout, NCW |
|
inline |
Computes 1D or 2D transposed convolution (aka fractionally strided convolution) of the input tensor. This operation can be seen as the gradient of Convolution operation with respect to its input. Convolution usually reduces the size of the input. Transposed convolution works the other way, going from a smaller.
data | Input tensor to the deconvolution operation. |
weight | Weights representing the kernel. |
bias | Bias added to the result after the deconvolution operation. |
kernel | Deconvolution kernel size: (w,), (h, w) or (d, h, w). This is same as |
num_filter | Number of output filters. |
stride | The stride used for the corresponding convolution: (w,), (h, w) or (d, |
dilate | Dilation factor for each dimension of the input: (w,), (h, w) or (d, h, |
pad | The amount of implicit zero padding added during convolution for each dimension of the input: (w,), (h, w) or (d, h, w). (kernel-1)/2 is usually a good choice. If target_shape is set, pad will be ignored and a padding |
adj | Adjustment for output shape: (w,), (h, w) or (d, h, w). If target_shape |
target_shape | Shape of the output tensor: (w,), (h, w) or (d, h, w). |
num_group | Number of groups partition. |
workspace | Maximum temporary workspace allowed (MB) in deconvolution.This parameter has two usages. When CUDNN is not used, it determines the effective batch size of the deconvolution kernel. When CUDNN is used, it controls the maximum temporary storage used for tuning the best CUDNN kernel when |
no_bias | Whether to disable bias parameter. |
cudnn_tune | Whether to pick convolution algorithm by running performance test. |
cudnn_off | Turn off cudnn for this layer. |
layout | Set layout for input, output and weight. Empty for default layout, NCW |
Converts each element of the input array from radians to degrees.
.. math:: degrees([0, /2, , 3/2, 2]) = [0, 90, 180, 270, 360]
The storage type of degrees
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L219
symbol_name | name of the resulting symbol |
data | The input array. |
Converts each element of the input array from radians to degrees.
.. math:: degrees([0, /2, , 3/2, 2]) = [0, 90, 180, 270, 360]
The storage type of degrees
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_trig.cc:L219
data | The input array. |
|
inline |
Rearranges(permutes) data from depth into blocks of spatial data. Similar to ONNX DepthToSpace operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#DepthToSpace. The output is a new tensor where the values from depth dimension are moved in to height and width dimension. The reverse of this operation is.
.. math::
{gather*} x = reshape(x, [N, block_size, block_size, C / (block_size ^ 2), H * x = transpose(x , [0, 3, 4, 1, 5, 2]) \ y = reshape(x , [N, C / (block_size ^ 2), H * block_size, W * {gather*}
where :math:x
is an input tensor with default layout as :math:[N, C, H, W]
: and :math:y
is the output tensor of layout :math:`[N, C / (block_size ^ 2),
Example::
x = [[[[0, 1, 2], [3, 4, 5]], [[6, 7, 8], [9, 10, 11]], [[12, 13, 14], [15, 16, 17]], [[18, 19, 20], [21, 22, 23]]]]
depth_to_space(x, 2) = [[[[0, 6, 1, 7, 2, 8], [12, 18, 13, 19, 14, 20], [3, 9, 4, 10, 5, 11], [15, 21, 16, 22, 17, 23]]]]
Defined in src/operator/tensor/matrix_op.cc:L1050
symbol_name | name of the resulting symbol |
data | Input ndarray |
block_size | Blocks of [block_size. block_size] are moved |
Rearranges(permutes) data from depth into blocks of spatial data. Similar to ONNX DepthToSpace operator: https://github.com/onnx/onnx/blob/master/docs/Operators.md#DepthToSpace. The output is a new tensor where the values from depth dimension are moved in to height and width dimension. The reverse of this operation is.
.. math::
{gather*} x = reshape(x, [N, block_size, block_size, C / (block_size ^ 2), H * x = transpose(x , [0, 3, 4, 1, 5, 2]) \ y = reshape(x , [N, C / (block_size ^ 2), H * block_size, W * {gather*}
where :math:x
is an input tensor with default layout as :math:[N, C, H, W]
: and :math:y
is the output tensor of layout :math:`[N, C / (block_size ^ 2),
Example::
x = [[[[0, 1, 2], [3, 4, 5]], [[6, 7, 8], [9, 10, 11]], [[12, 13, 14], [15, 16, 17]], [[18, 19, 20], [21, 22, 23]]]]
depth_to_space(x, 2) = [[[[0, 6, 1, 7, 2, 8], [12, 18, 13, 19, 14, 20], [3, 9, 4, 10, 5, 11], [15, 21, 16, 22, 17, 23]]]]
Defined in src/operator/tensor/matrix_op.cc:L1050
data | Input ndarray |
block_size | Blocks of [block_size. block_size] are moved |
|
inline |
Extracts a diagonal or constructs a diagonal array.
diag
's behavior depends on the input array dimensions:
axis1
and input shape and appending to the result a new axis with the size of theFor example, when the input shape is (2, 3, 4, 5)
, axis1
and axis2
respectively and k
is 0, the resulting shape would be (3, 5, 2)
.
Examples::
x = [[1, 2, 3], [4, 5, 6]]
diag(x) = [1, 5]
diag(x, k=1) = [2, 6]
diag(x, k=-1) = [4]
x = [1, 2, 3]
diag(x) = [[1, 0, 0], [0, 2, 0], [0, 0, 3]]
diag(x, k=1) = [[0, 1, 0], [0, 0, 2], [0, 0, 0]]
diag(x, k=-1) = [[0, 0, 0], [1, 0, 0], [0, 2, 0]]
x = [[[1, 2], [3, 4]],
[[5, 6], [7, 8]]]
diag(x) = [[1, 7], [2, 8]]
diag(x, k=1) = [[3], [4]]
diag(x, axis1=-2, axis2=-1) = [[1, 4], [5, 8]]
Defined in src/operator/tensor/diag_op.cc:L87
symbol_name | name of the resulting symbol |
data | Input ndarray |
k | Diagonal in question. The default is 0. Use k>0 for diagonals above the main diagonal, and k<0 for diagonals below the main diagonal. If input has shape (S0 |
axis1 | The first axis of the sub-arrays of interest. Ignored when the input is a |
axis2 | The second axis of the sub-arrays of interest. Ignored when the input is |
Extracts a diagonal or constructs a diagonal array.
diag
's behavior depends on the input array dimensions:
axis1
and input shape and appending to the result a new axis with the size of theFor example, when the input shape is (2, 3, 4, 5)
, axis1
and axis2
respectively and k
is 0, the resulting shape would be (3, 5, 2)
.
Examples::
x = [[1, 2, 3], [4, 5, 6]]
diag(x) = [1, 5]
diag(x, k=1) = [2, 6]
diag(x, k=-1) = [4]
x = [1, 2, 3]
diag(x) = [[1, 0, 0], [0, 2, 0], [0, 0, 3]]
diag(x, k=1) = [[0, 1, 0], [0, 0, 2], [0, 0, 0]]
diag(x, k=-1) = [[0, 0, 0], [1, 0, 0], [0, 2, 0]]
x = [[[1, 2], [3, 4]],
[[5, 6], [7, 8]]]
diag(x) = [[1, 7], [2, 8]]
diag(x, k=1) = [[3], [4]]
diag(x, axis1=-2, axis2=-1) = [[1, 4], [5, 8]]
Defined in src/operator/tensor/diag_op.cc:L87
data | Input ndarray |
k | Diagonal in question. The default is 0. Use k>0 for diagonals above the main diagonal, and k<0 for diagonals below the main diagonal. If input has shape (S0 |
axis1 | The first axis of the sub-arrays of interest. Ignored when the input is a |
axis2 | The second axis of the sub-arrays of interest. Ignored when the input is |
|
inline |
Dot product of two arrays.
dot
's behavior depends on the input array dimensions:
For example, given 3-D x
with shape (n,m,k)
and y
with shape result array will have shape (n,m,r,s)
. It is computed by::
dot(x,y)[i,j,a,b] = sum(x[i,j,:]*y[:,a,b])
Example::
x = reshape([0,1,2,3,4,5,6,7], shape=(2,2,2)) y = reshape([7,6,5,4,3,2,1,0], shape=(2,2,2)) dot(x,y)[0,0,1,1] = 0 sum(x[0,0,:]*y[:,1,1]) = 0
The storage type of dot
output depends on storage types of inputs, forward_stype option for output storage type. Implemented sparse operations
If the combination of input storage types and forward_stype does not match any above patterns, dot
will fallback and generate output with default storage.
.. Note::
If the storage type of the lhs is "csr", the storage type of gradient w.r.t rhs "row_sparse". Only a subset of optimizers support sparse gradients, including and Adam. Note that by default lazy updates is turned on, which may perform from standard updates. For more details, please check the Optimization API at: /api/python/optimization/optimization.html
Defined in src/operator/tensor/dot.cc:L77
symbol_name | name of the resulting symbol |
lhs | The first input |
rhs | The second input |
transpose_a | If true then transpose the first input before dot. |
transpose_b | If true then transpose the second input before dot. |
forward_stype | The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still |
|
inline |
Dot product of two arrays.
dot
's behavior depends on the input array dimensions:
For example, given 3-D x
with shape (n,m,k)
and y
with shape result array will have shape (n,m,r,s)
. It is computed by::
dot(x,y)[i,j,a,b] = sum(x[i,j,:]*y[:,a,b])
Example::
x = reshape([0,1,2,3,4,5,6,7], shape=(2,2,2)) y = reshape([7,6,5,4,3,2,1,0], shape=(2,2,2)) dot(x,y)[0,0,1,1] = 0 sum(x[0,0,:]*y[:,1,1]) = 0
The storage type of dot
output depends on storage types of inputs, forward_stype option for output storage type. Implemented sparse operations
If the combination of input storage types and forward_stype does not match any above patterns, dot
will fallback and generate output with default storage.
.. Note::
If the storage type of the lhs is "csr", the storage type of gradient w.r.t rhs "row_sparse". Only a subset of optimizers support sparse gradients, including and Adam. Note that by default lazy updates is turned on, which may perform from standard updates. For more details, please check the Optimization API at: /api/python/optimization/optimization.html
Defined in src/operator/tensor/dot.cc:L77
lhs | The first input |
rhs | The second input |
transpose_a | If true then transpose the first input before dot. |
transpose_b | If true then transpose the second input before dot. |
forward_stype | The desired storage type of the forward output given by user, if thecombination of input storage types and this hint does not matchany implemented ones, the dot operator will perform fallback operationand still |
|
inline |
Applies dropout operation to input array.
1/(1-p)
to keep the expected sum of the input unchanged.Example::
random.seed(998) input_array = array([[3., 0.5, -0.5, 2., 7.], [2., -0.4, 7., 3., 0.2]]) a = symbol.Variable('a') dropout = symbol.Dropout(a, p = 0.2) executor = dropout.simple_bind(a = input_array.shape)
executor.forward(is_train = True, a = input_array) executor.outputs [[ 3.75 0.625 -0. 2.5 8.75 ] [ 2.5 -0.5 8.75 3.75 0. ]]
executor.forward(is_train = False, a = input_array) executor.outputs [[ 3. 0.5 -0.5 2. 7. ] [ 2. -0.4 7. 3. 0.2 ]]
Defined in src/operator/nn/dropout.cc:L95
symbol_name | name of the resulting symbol |
data | Input array to which dropout will be applied. |
p | Fraction of the input that gets dropped out during training time. |
mode | Whether to only turn on dropout during training or to also turn on for |
axes | Axes for variational dropout kernel. |
cudnn_off | Whether to turn off cudnn in dropout operator. This option is ignored |
|
inline |
Applies dropout operation to input array.
1/(1-p)
to keep the expected sum of the input unchanged.Example::
random.seed(998) input_array = array([[3., 0.5, -0.5, 2., 7.], [2., -0.4, 7., 3., 0.2]]) a = symbol.Variable('a') dropout = symbol.Dropout(a, p = 0.2) executor = dropout.simple_bind(a = input_array.shape)
executor.forward(is_train = True, a = input_array) executor.outputs [[ 3.75 0.625 -0. 2.5 8.75 ] [ 2.5 -0.5 8.75 3.75 0. ]]
executor.forward(is_train = False, a = input_array) executor.outputs [[ 3. 0.5 -0.5 2. 7. ] [ 2. -0.4 7. 3. 0.2 ]]
Defined in src/operator/nn/dropout.cc:L95
data | Input array to which dropout will be applied. |
p | Fraction of the input that gets dropped out during training time. |
mode | Whether to only turn on dropout during training or to also turn on for |
axes | Axes for variational dropout kernel. |
cudnn_off | Whether to turn off cudnn in dropout operator. This option is ignored |
Adds arguments element-wise.
The storage type of elemwise_add
output depends on storage types of inputs
elemwise_add
generates output with default storagesymbol_name | name of the resulting symbol |
lhs | first input |
rhs | second input |
Adds arguments element-wise.
The storage type of elemwise_add
output depends on storage types of inputs
elemwise_add
generates output with default storagelhs | first input |
rhs | second input |
Divides arguments element-wise.
The storage type of elemwise_div
output is always dense
symbol_name | name of the resulting symbol |
lhs | first input |
rhs | second input |
Divides arguments element-wise.
The storage type of elemwise_div
output is always dense
lhs | first input |
rhs | second input |
Multiplies arguments element-wise.
The storage type of elemwise_mul
output depends on storage types of inputs
elemwise_mul
generates output with default storagesymbol_name | name of the resulting symbol |
lhs | first input |
rhs | second input |
Multiplies arguments element-wise.
The storage type of elemwise_mul
output depends on storage types of inputs
elemwise_mul
generates output with default storagelhs | first input |
rhs | second input |
Subtracts arguments element-wise.
The storage type of elemwise_sub
output depends on storage types of inputs
elemwise_sub
generates output with default storagesymbol_name | name of the resulting symbol |
lhs | first input |
rhs | second input |
Subtracts arguments element-wise.
The storage type of elemwise_sub
output depends on storage types of inputs
elemwise_sub
generates output with default storagelhs | first input |
rhs | second input |
|
inline |
Maps integer indices to vector representations (embeddings).
This operator maps words to real-valued vectors in a high-dimensional space, called word embeddings. These embeddings can capture semantic and syntactic For example, it has been noted that in the learned embedding spaces, similar to be close to each other and dissimilar words far apart.
For an input array of shape (d1, ..., dK), the shape of an output array is (d1, ..., dK, output_dim). All the input values should be integers in the range [0, input_dim).
If the input_dim is ip0 and output_dim is op0, then shape of the embedding (ip0, op0).
By default, if any index mentioned is too large, it is replaced by the index the last vector in an embedding matrix.
Examples::
input_dim = 4 output_dim = 5
// Each row in weight matrix y represents a word. So, y = (w0,w1,w2,w3) y = [[ 0., 1., 2., 3., 4.], [ 5., 6., 7., 8., 9.], [ 10., 11., 12., 13., 14.], [ 15., 16., 17., 18., 19.]]
// Input array x represents n-grams(2-gram). So, x = [(w1,w3), (w0,w2)] x = [[ 1., 3.], [ 0., 2.]]
// Mapped input x to its vector representation y. Embedding(x, y, 4, 5) = [[[ 5., 6., 7., 8., 9.], [ 15., 16., 17., 18., 19.]],
[[ 0., 1., 2., 3., 4.], [ 10., 11., 12., 13., 14.]]]
The storage type of weight can be either row_sparse or default. .. Note:: If "sparse_grad" is set to True, the storage type of gradient w.r.t weights "row_sparse". Only a subset of optimizers support sparse gradients, including and Adam. Note that by default lazy updates is turned on, which may perform from standard updates. For more details, please check the Optimization API at: /api/python/optimization/optimization.html Defined in src/operator/tensor/indexing_op.cc:L519
symbol_name | name of the resulting symbol |
data | The input array to the embedding operator. |
weight | The embedding weight matrix. |
input_dim | Vocabulary size of the input indices. |
output_dim | Dimension of the embedding vectors. |
dtype | Data type of weight. |
sparse_grad | Compute row sparse gradient in the backward calculation. If set to |
|
inline |
Maps integer indices to vector representations (embeddings).
This operator maps words to real-valued vectors in a high-dimensional space, called word embeddings. These embeddings can capture semantic and syntactic For example, it has been noted that in the learned embedding spaces, similar to be close to each other and dissimilar words far apart.
For an input array of shape (d1, ..., dK), the shape of an output array is (d1, ..., dK, output_dim). All the input values should be integers in the range [0, input_dim).
If the input_dim is ip0 and output_dim is op0, then shape of the embedding (ip0, op0).
By default, if any index mentioned is too large, it is replaced by the index the last vector in an embedding matrix.
Examples::
input_dim = 4 output_dim = 5
// Each row in weight matrix y represents a word. So, y = (w0,w1,w2,w3) y = [[ 0., 1., 2., 3., 4.], [ 5., 6., 7., 8., 9.], [ 10., 11., 12., 13., 14.], [ 15., 16., 17., 18., 19.]]
// Input array x represents n-grams(2-gram). So, x = [(w1,w3), (w0,w2)] x = [[ 1., 3.], [ 0., 2.]]
// Mapped input x to its vector representation y. Embedding(x, y, 4, 5) = [[[ 5., 6., 7., 8., 9.], [ 15., 16., 17., 18., 19.]],
[[ 0., 1., 2., 3., 4.], [ 10., 11., 12., 13., 14.]]]
The storage type of weight can be either row_sparse or default. .. Note:: If "sparse_grad" is set to True, the storage type of gradient w.r.t weights "row_sparse". Only a subset of optimizers support sparse gradients, including and Adam. Note that by default lazy updates is turned on, which may perform from standard updates. For more details, please check the Optimization API at: /api/python/optimization/optimization.html Defined in src/operator/tensor/indexing_op.cc:L519
data | The input array to the embedding operator. |
weight | The embedding weight matrix. |
input_dim | Vocabulary size of the input indices. |
output_dim | Dimension of the embedding vectors. |
dtype | Data type of weight. |
sparse_grad | Compute row sparse gradient in the backward calculation. If set to |
Returns element-wise gauss error function of the input.
Example::
erf([0, -1., 10.]) = [0., -0.8427, 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L964
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise gauss error function of the input.
Example::
erf([0, -1., 10.]) = [0., -0.8427, 1.]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L964
data | The input array. |
Returns element-wise inverse gauss error function of the input.
Example::
erfinv([0, 0.5., -1.]) = [0., 0.4769, -inf]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L985
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise inverse gauss error function of the input.
Example::
erfinv([0, 0.5., -1.]) = [0., 0.4769, -inf]
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L985
data | The input array. |
Returns element-wise exponential value of the input.
.. math:: exp(x) = e^x 2.718^x
Example::
exp([0, 1, 2]) = [1., 2.71828175, 7.38905621]
The storage type of exp
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1044
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise exponential value of the input.
.. math:: exp(x) = e^x 2.718^x
Example::
exp([0, 1, 2]) = [1., 2.71828175, 7.38905621]
The storage type of exp
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1044
data | The input array. |
Inserts a new axis of size 1 into the array shape.
For example, given x
with shape (2,3,4)
, then expand_dims(x, axis=1)
will return a new array with shape (2,1,3,4)
.
Defined in src/operator/tensor/matrix_op.cc:L416
symbol_name | name of the resulting symbol |
data | Source input |
axis | Position where new axis is to be inserted. Suppose that the input NDArray 's dimension is ndim , the range of the inserted axis is `[-ndim, |
Inserts a new axis of size 1 into the array shape.
For example, given x
with shape (2,3,4)
, then expand_dims(x, axis=1)
will return a new array with shape (2,1,3,4)
.
Defined in src/operator/tensor/matrix_op.cc:L416
data | Source input |
axis | Position where new axis is to be inserted. Suppose that the input NDArray 's dimension is ndim , the range of the inserted axis is `[-ndim, |
Returns exp(x) - 1
computed element-wise on the input.
This function provides greater precision than exp(x) - 1
for small values
The storage type of expm1
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1189
symbol_name | name of the resulting symbol |
data | The input array. |
Returns exp(x) - 1
computed element-wise on the input.
This function provides greater precision than exp(x) - 1
for small values
The storage type of expm1
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1189
data | The input array. |
|
inline |
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function.
symbol_name | name of the resulting symbol |
lhs | Left operand to the function. |
mhs | Middle operand to the function. |
rhs | Right operand to the function. |
Fill one element of each line(row for python, column for R/Julia) in lhs according to index indicated by rhs and values indicated by mhs. This function.
lhs | Left operand to the function. |
mhs | Middle operand to the function. |
rhs | Right operand to the function. |
Returns element-wise rounded value to the nearest \ integer towards zero of the input.
Example::
fix([-2.1, -1.9, 1.9, 2.1]) = [-2., -1., 1., 2.]
The storage type of fix
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L843
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise rounded value to the nearest \ integer towards zero of the input.
Example::
fix([-2.1, -1.9, 1.9, 2.1]) = [-2., -1., 1., 2.]
The storage type of fix
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L843
data | The input array. |
Flattens the input array into a 2-D array by collapsing the higher dimensions.
.. note:: Flatten
is deprecated. Use flatten
instead.
For an input array with shape (d1, d2, ..., dk)
, flatten
operation the input array into an output array of shape (d1, d2*...*dk)
.
Note that the bahavior of this function is different from numpy.ndarray.flatten, which behaves similar to mxnet.ndarray.reshape((-1,)).
Example::
x = [[ [1,2,3], [4,5,6], [7,8,9] ], [ [1,2,3], [4,5,6], [7,8,9] ]],
flatten(x) = [[ 1., 2., 3., 4., 5., 6., 7., 8., 9.], [ 1., 2., 3., 4., 5., 6., 7., 8., 9.]]
Defined in src/operator/tensor/matrix_op.cc:L291
symbol_name | name of the resulting symbol |
data | Input array. |
Flattens the input array into a 2-D array by collapsing the higher dimensions.
.. note:: Flatten
is deprecated. Use flatten
instead.
For an input array with shape (d1, d2, ..., dk)
, flatten
operation the input array into an output array of shape (d1, d2*...*dk)
.
Note that the bahavior of this function is different from numpy.ndarray.flatten, which behaves similar to mxnet.ndarray.reshape((-1,)).
Example::
x = [[ [1,2,3], [4,5,6], [7,8,9] ], [ [1,2,3], [4,5,6], [7,8,9] ]],
flatten(x) = [[ 1., 2., 3., 4., 5., 6., 7., 8., 9.], [ 1., 2., 3., 4., 5., 6., 7., 8., 9.]]
Defined in src/operator/tensor/matrix_op.cc:L291
data | Input array. |
Returns element-wise floor of the input.
The floor of the scalar x is the largest integer i, such that i <= x.
Example::
floor([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-3., -2., 1., 1., 2.]
The storage type of floor
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L805
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise floor of the input.
The floor of the scalar x is the largest integer i, such that i <= x.
Example::
floor([-2.1, -1.9, 1.5, 1.9, 2.1]) = [-3., -2., 1., 1., 2.]
The storage type of floor
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L805
data | The input array. |
|
inline |
The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
.. math::
g_t = J(W_{t-1})\ v_t = v_{t-1} + (1 - ) g_t^2\ d_t = { 1 - ^t }{ } ({ { v_t }{ 1 - ^t } } = d_t - d_{t-1} z_t = z_{ t-1 } + (1 - ^t) g_t - W_{t-1} W_t = - { z_t }{ d_t }
Defined in src/operator/optimizer_op.cc:L638
symbol_name | name of the resulting symbol |
weight | Weight |
grad | Gradient |
d | Internal state d_t |
v | Internal state v_t |
z | Internal state z_t |
lr | Learning rate. |
t | Number of update. |
beta1 | Generally close to 0.5. |
beta2 | Generally close to 1. |
epsilon | Epsilon to prevent div 0. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_grad | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
|
inline |
The FTML optimizer described in FTML - Follow the Moving Leader in Deep Learning, available at http://proceedings.mlr.press/v70/zheng17a/zheng17a.pdf.
.. math::
g_t = J(W_{t-1})\ v_t = v_{t-1} + (1 - ) g_t^2\ d_t = { 1 - ^t }{ } ({ { v_t }{ 1 - ^t } } = d_t - d_{t-1} z_t = z_{ t-1 } + (1 - ^t) g_t - W_{t-1} W_t = - { z_t }{ d_t }
Defined in src/operator/optimizer_op.cc:L638
weight | Weight |
grad | Gradient |
d | Internal state d_t |
v | Internal state v_t |
z | Internal state z_t |
lr | Learning rate. |
t | Number of update. |
beta1 | Generally close to 0.5. |
beta2 | Generally close to 1. |
epsilon | Epsilon to prevent div 0. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_grad | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
|
inline |
Update function for Ftrl optimizer. Referenced from Ad Click Prediction: a View from the Trenches, available at http://dl.acm.org/citation.cfm?id=2488200.
It updates the weights using::
rescaled_grad = clip(grad * rescale_grad, clip_gradient) z += rescaled_grad - (sqrt(n + rescaled_grad**2) - sqrt(n)) * weight / n += rescaled_grad**2 w = (sign(z) * lamda1 - z) / ((beta + sqrt(n)) / learning_rate + wd) * (abs(z)
If w, z and n are all of row_sparse
storage type, only the row slices whose indices appear in grad.indices are updated (for w, z
for row in grad.indices: rescaled_grad[row] = clip(grad[row] * rescale_grad, clip_gradient) z[row] += rescaled_grad[row] - (sqrt(n[row] + rescaled_grad[row]**2) - n[row] += rescaled_grad[row]**2 w[row] = (sign(z[row]) * lamda1 - z[row]) / ((beta + sqrt(n[row])) /
Defined in src/operator/optimizer_op.cc:L874
symbol_name | name of the resulting symbol |
weight | Weight |
grad | Gradient |
z | z |
n | Square of grad |
lr | Learning rate |
lamda1 | The L1 regularization coefficient. |
beta | Per-Coordinate Learning Rate beta. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_gradient | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
|
inline |
Update function for Ftrl optimizer. Referenced from Ad Click Prediction: a View from the Trenches, available at http://dl.acm.org/citation.cfm?id=2488200.
It updates the weights using::
rescaled_grad = clip(grad * rescale_grad, clip_gradient) z += rescaled_grad - (sqrt(n + rescaled_grad**2) - sqrt(n)) * weight / n += rescaled_grad**2 w = (sign(z) * lamda1 - z) / ((beta + sqrt(n)) / learning_rate + wd) * (abs(z)
If w, z and n are all of row_sparse
storage type, only the row slices whose indices appear in grad.indices are updated (for w, z
for row in grad.indices: rescaled_grad[row] = clip(grad[row] * rescale_grad, clip_gradient) z[row] += rescaled_grad[row] - (sqrt(n[row] + rescaled_grad[row]**2) - n[row] += rescaled_grad[row]**2 w[row] = (sign(z[row]) * lamda1 - z[row]) / ((beta + sqrt(n[row])) /
Defined in src/operator/optimizer_op.cc:L874
weight | Weight |
grad | Gradient |
z | z |
n | Square of grad |
lr | Learning rate |
lamda1 | The L1 regularization coefficient. |
beta | Per-Coordinate Learning Rate beta. |
wd | Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of |
rescale_grad | Rescale gradient to grad = rescale_grad*grad. |
clip_gradient | Clip gradient to the range of [-clip_gradient, clip_gradient] If clip_gradient <= 0, gradient clipping is turned off. grad = max(min(grad, |
|
inline |
Applies a linear transformation: :math:Y = XW^T + b
.
If flatten
is set to be true, then the shapes are:
(batch_size, x1, x2, ..., xn)
(num_hidden, x1 * x2 * ... * xn)
(num_hidden,)
(batch_size, num_hidden)
If flatten
is set to be false, then the shapes are:
(x1, x2, ..., xn, input_dim)
(num_hidden, input_dim)
(num_hidden,)
(x1, x2, ..., xn, num_hidden)
The learnable parameters include both weight
and bias
.
If no_bias
is set to be true, then the bias
term is ignored.
.. Note::
The sparse support for FullyConnected is limited to forward evaluation with weight and bias, where the length of weight.indices
and bias.indices
must to num_hidden
. This could be useful for model inference with row_sparse
trained with importance sampling or noise contrastive estimation.
To compute linear transformation with 'csr' sparse data, sparse.dot is of sparse.FullyConnected.
Defined in src/operator/nn/fully_connected.cc:L277
symbol_name | name of the resulting symbol |
data | Input data. |
weight | Weight matrix. |
bias | Bias parameter. |
num_hidden | Number of hidden nodes of the output. |
no_bias | Whether to disable bias parameter. |
flatten | Whether to collapse all but the first axis of the input data tensor. |
|
inline |
Applies a linear transformation: :math:Y = XW^T + b
.
If flatten
is set to be true, then the shapes are:
(batch_size, x1, x2, ..., xn)
(num_hidden, x1 * x2 * ... * xn)
(num_hidden,)
(batch_size, num_hidden)
If flatten
is set to be false, then the shapes are:
(x1, x2, ..., xn, input_dim)
(num_hidden, input_dim)
(num_hidden,)
(x1, x2, ..., xn, num_hidden)
The learnable parameters include both weight
and bias
.
If no_bias
is set to be true, then the bias
term is ignored.
.. Note::
The sparse support for FullyConnected is limited to forward evaluation with weight and bias, where the length of weight.indices
and bias.indices
must to num_hidden
. This could be useful for model inference with row_sparse
trained with importance sampling or noise contrastive estimation.
To compute linear transformation with 'csr' sparse data, sparse.dot is of sparse.FullyConnected.
Defined in src/operator/nn/fully_connected.cc:L277
data | Input data. |
weight | Weight matrix. |
bias | Bias parameter. |
num_hidden | Number of hidden nodes of the output. |
no_bias | Whether to disable bias parameter. |
flatten | Whether to collapse all but the first axis of the input data tensor. |
Returns the gamma function (extension of the factorial function \ to the reals), computed element-wise on the input array.
The storage type of gamma
output is always dense
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the gamma function (extension of the factorial function \ to the reals), computed element-wise on the input array.
The storage type of gamma
output is always dense
data | The input array. |
Returns element-wise log of the absolute value of the gamma function \ of the input.
The storage type of gammaln
output is always dense
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise log of the absolute value of the gamma function \ of the input.
The storage type of gammaln
output is always dense
data | The input array. |
|
inline |
Gather elements or slices from data
and store to a tensor whose shape is defined by indices
.
Given data
with shape (X_0, X_1, ..., X_{N-1})
and indices with shape (M, Y_0, ..., Y_{K-1})
, the output will have shape (Y_0, ..., Y_{K-1}, X_M, where
M <= N. If
M == N, output shape will simply be
(Y_0, ..., Y_{K-1})`.
The elements in output is defined as follows::
output[y_0, ..., y_{K-1}, x_M, ..., x_{N-1}] = data[indices[0, y_0, ..., ..., indices[M-1, y_0, ..., y_{K-1}], x_M, ..., x_{N-1}]
Examples::
data = [[0, 1], [2, 3]] indices = [[1, 1, 0], [0, 1, 0]] gather_nd(data, indices) = [2, 3, 0]
data = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]] indices = [[0, 1], [1, 0]] gather_nd(data, indices) = [[3, 4], [5, 6]]
symbol_name | name of the resulting symbol |
data | data |
indices | indices |
Gather elements or slices from data
and store to a tensor whose shape is defined by indices
.
Given data
with shape (X_0, X_1, ..., X_{N-1})
and indices with shape (M, Y_0, ..., Y_{K-1})
, the output will have shape (Y_0, ..., Y_{K-1}, X_M, where
M <= N. If
M == N, output shape will simply be
(Y_0, ..., Y_{K-1})`.
The elements in output is defined as follows::
output[y_0, ..., y_{K-1}, x_M, ..., x_{N-1}] = data[indices[0, y_0, ..., ..., indices[M-1, y_0, ..., y_{K-1}], x_M, ..., x_{N-1}]
Examples::
data = [[0, 1], [2, 3]] indices = [[1, 1, 0], [0, 1, 0]] gather_nd(data, indices) = [2, 3, 0]
data = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]] indices = [[0, 1], [1, 0]] gather_nd(data, indices) = [[3, 4], [5, 6]]
data | data |
indices | indices |
|
inline |
Generates 2D sampling grid for bilinear sampling.
symbol_name | name of the resulting symbol |
data | Input data to the function. |
transform_type | The type of transformation. For affine , input data should be an affine matrix of size (batch, 6). For warp , input data should be an |
target_shape | Specifies the output shape (H, W). This is required if transformation type is affine . If transformation type is warp , this |
|
inline |
Generates 2D sampling grid for bilinear sampling.
data | Input data to the function. |
transform_type | The type of transformation. For affine , input data should be an affine matrix of size (batch, 6). For warp , input data should be an |
target_shape | Specifies the output shape (H, W). This is required if transformation type is affine . If transformation type is warp , this |
|
inline |
Computes hard sigmoid of x element-wise.
.. math:: y = max(0, min(1, alpha * x + beta))
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L133
symbol_name | name of the resulting symbol |
data | The input array. |
alpha | Slope of hard sigmoid |
beta | Bias of hard sigmoid. |
|
inline |
Computes hard sigmoid of x element-wise.
.. math:: y = max(0, min(1, alpha * x + beta))
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L133
data | The input array. |
alpha | Slope of hard sigmoid |
beta | Bias of hard sigmoid. |
|
inline |
Apply a sparse regularization to the output a sigmoid activation function.
symbol_name | name of the resulting symbol |
data | Input data. |
sparseness_target | The sparseness target |
penalty | The tradeoff parameter for the sparseness penalty |
momentum | The momentum for running average |
|
inline |
Apply a sparse regularization to the output a sigmoid activation function.
data | Input data. |
sparseness_target | The sparseness target |
penalty | The tradeoff parameter for the sparseness penalty |
momentum | The momentum for running average |
|
inline |
Applies instance normalization to the n-dimensional input array.
This operator takes an n-dimensional input array where (n>2) and normalizes the input using the following formula:
.. math::
out = {x - mean[data]}{ {Var[data]} + } * gamma + beta
This layer is similar to batch normalization layer (BatchNorm
) with two differences: first, the normalization is carried out per example (instance), not over a batch. Second, the same normalization is applied both at test and train time. This operation is also known as contrast normalization
.
If the input data is of shape [batch, channel, spacial_dim1, spacial_dim2, ...], gamma
and beta
parameters must be vectors of shape [channel].
This implementation is based on paper:
.. [1] Instance Normalization: The Missing Ingredient for Fast Stylization, D. Ulyanov, A. Vedaldi, V. Lempitsky, 2016 (arXiv:1607.08022v2).
Examples::
// Input of shape (2,1,2) x = [[[ 1.1, 2.2]], [[ 3.3, 4.4]]]
// gamma parameter of length 1 gamma = [1.5]
// beta parameter of length 1 beta = [0.5]
// Instance normalization is calculated with the above formula InstanceNorm(x,gamma,beta) = [[[-0.997527 , 1.99752665]], [[-0.99752653, 1.99752724]]]
Defined in src/operator/instance_norm.cc:L95
symbol_name | name of the resulting symbol |
data | An n-dimensional input array (n > 2) of the form [batch, channel, |
gamma | A vector of length 'channel', which multiplies the normalized input. |
beta | A vector of length 'channel', which is added to the product of the |
eps | An epsilon parameter to prevent division by 0. |
|
inline |
Applies instance normalization to the n-dimensional input array.
This operator takes an n-dimensional input array where (n>2) and normalizes the input using the following formula:
.. math::
out = {x - mean[data]}{ {Var[data]} + } * gamma + beta
This layer is similar to batch normalization layer (BatchNorm
) with two differences: first, the normalization is carried out per example (instance), not over a batch. Second, the same normalization is applied both at test and train time. This operation is also known as contrast normalization
.
If the input data is of shape [batch, channel, spacial_dim1, spacial_dim2, ...], gamma
and beta
parameters must be vectors of shape [channel].
This implementation is based on paper:
.. [1] Instance Normalization: The Missing Ingredient for Fast Stylization, D. Ulyanov, A. Vedaldi, V. Lempitsky, 2016 (arXiv:1607.08022v2).
Examples::
// Input of shape (2,1,2) x = [[[ 1.1, 2.2]], [[ 3.3, 4.4]]]
// gamma parameter of length 1 gamma = [1.5]
// beta parameter of length 1 beta = [0.5]
// Instance normalization is calculated with the above formula InstanceNorm(x,gamma,beta) = [[[-0.997527 , 1.99752665]], [[-0.99752653, 1.99752724]]]
Defined in src/operator/instance_norm.cc:L95
data | An n-dimensional input array (n > 2) of the form [batch, channel, |
gamma | A vector of length 'channel', which multiplies the normalized input. |
beta | A vector of length 'channel', which is added to the product of the |
eps | An epsilon parameter to prevent division by 0. |
|
inline |
Computes the Khatri-Rao product of the input matrices.
Given a collection of :math:n
input matrices,
.. math:: A_1 {R}^{M_1 M}, , A_n {R}^{M_n N},
the (column-wise) Khatri-Rao product is defined as the matrix,
.. math:: X = A_1 A_n {R}^{(M_1 M_n) N},
where the :math:k
th column is equal to the column-wise outer product :math:{A_1}_k \otimes \cdots \otimes {A_n}_k
where :math:{A_i}_k
is the kth column of the ith matrix.
Example::
>>> A = mx.nd.array([[1, -1], >>> [2, -3]]) >>> B = mx.nd.array([[1, 4], >>> [2, 5], >>> [3, 6]]) >>> C = mx.nd.khatri_rao(A, B) >>> print(C.asnumpy()) [[ 1. -4.] [ 2. -5.] [ 3. -6.] [ 2. -12.] [ 4. -15.] [ 6. -18.]]
Defined in src/operator/contrib/krprod.cc:L108
symbol_name | name of the resulting symbol |
args | Positional input matrices |
Computes the Khatri-Rao product of the input matrices.
Given a collection of :math:n
input matrices,
.. math:: A_1 {R}^{M_1 M}, , A_n {R}^{M_n N},
the (column-wise) Khatri-Rao product is defined as the matrix,
.. math:: X = A_1 A_n {R}^{(M_1 M_n) N},
where the :math:k
th column is equal to the column-wise outer product :math:{A_1}_k \otimes \cdots \otimes {A_n}_k
where :math:{A_i}_k
is the kth column of the ith matrix.
Example::
>>> A = mx.nd.array([[1, -1], >>> [2, -3]]) >>> B = mx.nd.array([[1, 4], >>> [2, 5], >>> [3, 6]]) >>> C = mx.nd.khatri_rao(A, B) >>> print(C.asnumpy()) [[ 1. -4.] [ 2. -5.] [ 3. -6.] [ 2. -12.] [ 4. -15.] [ 6. -18.]]
Defined in src/operator/contrib/krprod.cc:L108
args | Positional input matrices |
|
inline |
Normalize the input array using the L2 norm.
For 1-D NDArray, it computes::
out = data / sqrt(sum(data ** 2) + eps)
For N-D NDArray, if the input array has shape (N, N, ..., N),
with mode
= instance
, it normalizes each instance in the array by its L2 norm.::
for i in 0...N out[i,:,:,...,:] = data[i,:,:,...,:] / sqrt(sum(data[i,:,:,...,:] ** 2) + eps)
with mode
= channel
, it normalizes each channel in the array by its L2
for i in 0...N out[:,i,:,...,:] = data[:,i,:,...,:] / sqrt(sum(data[:,i,:,...,:] ** 2) + eps)
with mode
= spatial
, it normalizes the cross channel norm for each in the array by its L2 norm.::
for dim in 2...N for i in 0...N out[.....,i,...] = take(out, indices=i, axis=dim) / sqrt(sum(take(out, -dim-
Example::
x = [[[1,2], [3,4]], [[2,2], [5,6]]]
L2Normalization(x, mode='instance') =[[[ 0.18257418 0.36514837] [ 0.54772252 0.73029673]] [[ 0.24077171 0.24077171] [ 0.60192931 0.72231513]]]
L2Normalization(x, mode='channel') =[[[ 0.31622776 0.44721359] [ 0.94868326 0.89442718]] [[ 0.37139067 0.31622776] [ 0.92847669 0.94868326]]]
L2Normalization(x, mode='spatial') =[[[ 0.44721359 0.89442718] [ 0.60000002 0.80000001]] [[ 0.70710677 0.70710677] [ 0.6401844 0.76822126]]]
Defined in src/operator/l2_normalization.cc:L196
symbol_name | name of the resulting symbol |
data | Input array to normalize. |
eps | A small constant for numerical stability. |
mode | Specify the dimension along which to compute L2 norm. |
|
inline |
Normalize the input array using the L2 norm.
For 1-D NDArray, it computes::
out = data / sqrt(sum(data ** 2) + eps)
For N-D NDArray, if the input array has shape (N, N, ..., N),
with mode
= instance
, it normalizes each instance in the array by its L2 norm.::
for i in 0...N out[i,:,:,...,:] = data[i,:,:,...,:] / sqrt(sum(data[i,:,:,...,:] ** 2) + eps)
with mode
= channel
, it normalizes each channel in the array by its L2
for i in 0...N out[:,i,:,...,:] = data[:,i,:,...,:] / sqrt(sum(data[:,i,:,...,:] ** 2) + eps)
with mode
= spatial
, it normalizes the cross channel norm for each in the array by its L2 norm.::
for dim in 2...N for i in 0...N out[.....,i,...] = take(out, indices=i, axis=dim) / sqrt(sum(take(out, -dim-
Example::
x = [[[1,2], [3,4]], [[2,2], [5,6]]]
L2Normalization(x, mode='instance') =[[[ 0.18257418 0.36514837] [ 0.54772252 0.73029673]] [[ 0.24077171 0.24077171] [ 0.60192931 0.72231513]]]
L2Normalization(x, mode='channel') =[[[ 0.31622776 0.44721359] [ 0.94868326 0.89442718]] [[ 0.37139067 0.31622776] [ 0.92847669 0.94868326]]]
L2Normalization(x, mode='spatial') =[[[ 0.44721359 0.89442718] [ 0.60000002 0.80000001]] [[ 0.70710677 0.70710677] [ 0.6401844 0.76822126]]]
Defined in src/operator/l2_normalization.cc:L196
data | Input array to normalize. |
eps | A small constant for numerical stability. |
mode | Specify the dimension along which to compute L2 norm. |
|
inline |
Layer normalization.
Normalizes the channels of the input tensor by mean and variance, and applies a well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis and then compute the normalized output, which has the same shape as input, as following:
.. math::
out = {data - mean(data, axis)}{{var(data, axis) + }} * gamma
Both gamma
and beta
are learnable parameters.
Unlike BatchNorm and InstanceNorm, the mean and var are computed along the
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both data_std
. Note that no gradient will be passed through these two outputs.
The parameter axis
specifies which axis of the input shape denotes the 'channel' (separately normalized groups). The default is -1, which sets axis to be the last item in the input shape.
Defined in src/operator/nn/layer_norm.cc:L155
symbol_name | name of the resulting symbol |
data | Input data to layer normalization |
gamma | gamma array |
beta | beta array |
axis | The axis to perform layer normalization. Usually, this should be be axis |
eps | An epsilon parameter to prevent division by 0. |
output_mean_var | Output the mean and std calculated along the given axis. |
|
inline |
Layer normalization.
Normalizes the channels of the input tensor by mean and variance, and applies a well as offset beta
.
Assume the input has more than one dimension and we normalize along axis 1. We first compute the mean and variance along this axis and then compute the normalized output, which has the same shape as input, as following:
.. math::
out = {data - mean(data, axis)}{{var(data, axis) + }} * gamma
Both gamma
and beta
are learnable parameters.
Unlike BatchNorm and InstanceNorm, the mean and var are computed along the
Assume the input has size k on axis 1, then both gamma
and beta
have shape *(k,)*. If output_mean_var
is set to be true, then outputs both data_std
. Note that no gradient will be passed through these two outputs.
The parameter axis
specifies which axis of the input shape denotes the 'channel' (separately normalized groups). The default is -1, which sets axis to be the last item in the input shape.
Defined in src/operator/nn/layer_norm.cc:L155
data | Input data to layer normalization |
gamma | gamma array |
beta | beta array |
axis | The axis to perform layer normalization. Usually, this should be be axis |
eps | An epsilon parameter to prevent division by 0. |
output_mean_var | Output the mean and std calculated along the given axis. |
|
inline |
Applies Leaky rectified linear unit activation element-wise to the input.
Leaky ReLUs attempt to fix the "dying ReLU" problem by allowing a small slope
when the input is negative and has a slope of one when input is positive.
The following modified ReLU Activation functions are supported:
y = x > 0 ? x : slope * (exp(x)-1)
y = x > 0 ? x : slope * x
slope
isslope
is uniformly and [lower_bound, upper_bound) for training, while fixed to be *(lower_bound+upper_bound)/2* for inference.Defined in src/operator/leaky_relu.cc:L65
symbol_name | name of the resulting symbol |
data | Input data to activation function. |
gamma | Slope parameter for PReLU. Only required when act_type is 'prelu'. It should be either a vector of size 1, or the same size as the second dimension |
act_type | Activation function to be applied. |
slope | Init slope for the activation. (For leaky and elu only) |
lower_bound | Lower bound of random slope. (For rrelu only) |
upper_bound | Upper bound of random slope. (For rrelu only) |
|
inline |
Applies Leaky rectified linear unit activation element-wise to the input.
Leaky ReLUs attempt to fix the "dying ReLU" problem by allowing a small slope
when the input is negative and has a slope of one when input is positive.
The following modified ReLU Activation functions are supported:
y = x > 0 ? x : slope * (exp(x)-1)
y = x > 0 ? x : slope * x
slope
isslope
is uniformly and [lower_bound, upper_bound) for training, while fixed to be *(lower_bound+upper_bound)/2* for inference.Defined in src/operator/leaky_relu.cc:L65
data | Input data to activation function. |
gamma | Slope parameter for PReLU. Only required when act_type is 'prelu'. It should be either a vector of size 1, or the same size as the second dimension |
act_type | Activation function to be applied. |
slope | Init slope for the activation. (For leaky and elu only) |
lower_bound | Lower bound of random slope. (For rrelu only) |
upper_bound | Upper bound of random slope. (For rrelu only) |
|
inline |
Computes and optimizes for squared loss during backward propagation. Just outputs data
during forward propagation.
If :math:\hat{y}_i
is the predicted value of the i-th sample, and :math:y_i
then the squared loss estimated over :math:n
samples is defined as
:math:`{SquaredLoss}({Y}, {{Y}} ) = {1}{n}
.. note:: Use the LinearRegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
By default, gradients of this loss function are scaled by factor 1/m
, where m The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L92
symbol_name | name of the resulting symbol |
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
|
inline |
Computes and optimizes for squared loss during backward propagation. Just outputs data
during forward propagation.
If :math:\hat{y}_i
is the predicted value of the i-th sample, and :math:y_i
then the squared loss estimated over :math:n
samples is defined as
:math:`{SquaredLoss}({Y}, {{Y}} ) = {1}{n}
.. note:: Use the LinearRegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
By default, gradients of this loss function are scaled by factor 1/m
, where m The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L92
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
Returns element-wise Natural logarithmic value of the input.
The natural logarithm is logarithm in base e, so that log(exp(x)) = x
The storage type of log
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1057
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise Natural logarithmic value of the input.
The natural logarithm is logarithm in base e, so that log(exp(x)) = x
The storage type of log
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1057
data | The input array. |
Returns element-wise Base-10 logarithmic value of the input.
10**log10(x) = x
The storage type of log10
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1074
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise Base-10 logarithmic value of the input.
10**log10(x) = x
The storage type of log10
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1074
data | The input array. |
Returns element-wise log(1 + x)
value of the input.
This function is more accurate than log(1 + x)
for small x
so that :math:1+x\approx 1
The storage type of log1p
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1171
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise log(1 + x)
value of the input.
This function is more accurate than log(1 + x)
for small x
so that :math:1+x\approx 1
The storage type of log1p
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1171
data | The input array. |
Returns element-wise Base-2 logarithmic value of the input.
2**log2(x) = x
The storage type of log2
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1086
symbol_name | name of the resulting symbol |
data | The input array. |
Returns element-wise Base-2 logarithmic value of the input.
2**log2(x) = x
The storage type of log2
output is always dense
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L1086
data | The input array. |
|
inline |
Computes the log softmax of the input. This is equivalent to computing softmax followed by log.
Examples::
>>> x = mx.nd.array([1, 2, .1]) >>> mx.nd.log_softmax(x).asnumpy() array([-1.41702998, -0.41702995, -2.31702995], dtype=float32)
>>> x = mx.nd.array( [[1, 2, .1],[.1, 2, 1]] ) >>> mx.nd.log_softmax(x, axis=0).asnumpy() array([[-0.34115392, -0.69314718, -1.24115396], [-1.24115396, -0.69314718, -0.34115392]], dtype=float32)
symbol_name | name of the resulting symbol |
data | The input array. |
axis | The axis along which to compute softmax. |
temperature | Temperature parameter in softmax |
dtype | DType of the output in case this can't be inferred. Defaults to the same |
|
inline |
Computes the log softmax of the input. This is equivalent to computing softmax followed by log.
Examples::
>>> x = mx.nd.array([1, 2, .1]) >>> mx.nd.log_softmax(x).asnumpy() array([-1.41702998, -0.41702995, -2.31702995], dtype=float32)
>>> x = mx.nd.array( [[1, 2, .1],[.1, 2, 1]] ) >>> mx.nd.log_softmax(x, axis=0).asnumpy() array([[-0.34115392, -0.69314718, -1.24115396], [-1.24115396, -0.69314718, -0.34115392]], dtype=float32)
data | The input array. |
axis | The axis along which to compute softmax. |
temperature | Temperature parameter in softmax |
dtype | DType of the output in case this can't be inferred. Defaults to the same |
Returns the result of logical NOT (!) function.
Example: logical_not([-2., 0., 1.]) = [0., 1., 0.]
symbol_name | name of the resulting symbol |
data | The input array. |
Returns the result of logical NOT (!) function.
Example: logical_not([-2., 0., 1.]) = [0., 1., 0.]
data | The input array. |
|
inline |
Applies a logistic function to the input.
The logistic function, also known as the sigmoid function, is computed as :math:\frac{1}{1+exp(-\textbf{x})}
.
Commonly, the sigmoid is used to squash the real-valued output of a linear model :math:wTx+b
into the [0,1] range so that it can be interpreted as a It is suitable for binary classification or probability prediction tasks.
.. note:: Use the LogisticRegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
The loss function used is the Binary Cross Entropy Loss:
:math:-{(y\log(p) + (1 - y)\log(1 - p))}
Where y
is the ground truth probability of positive outcome for a given example, and p
the probability predicted by the model. By default, gradients of this loss function are scaled by factor 1/m
, where m is the number of The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L152
symbol_name | name of the resulting symbol |
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
|
inline |
Applies a logistic function to the input.
The logistic function, also known as the sigmoid function, is computed as :math:\frac{1}{1+exp(-\textbf{x})}
.
Commonly, the sigmoid is used to squash the real-valued output of a linear model :math:wTx+b
into the [0,1] range so that it can be interpreted as a It is suitable for binary classification or probability prediction tasks.
.. note:: Use the LogisticRegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
The loss function used is the Binary Cross Entropy Loss:
:math:-{(y\log(p) + (1 - y)\log(1 - p))}
Where y
is the ground truth probability of positive outcome for a given example, and p
the probability predicted by the model. By default, gradients of this loss function are scaled by factor 1/m
, where m is the number of The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L152
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
|
inline |
Applies local response normalization to the input.
The local response normalization layer performs "lateral inhibition" by over local input regions.
If :math:a_{x,y}^{i}
is the activity of a neuron computed by applying kernel :math:(x, y)
and then applying the ReLU nonlinearity, the response-normalized activity :math:b_{x,y}^{i}
is given by the expression:
.. math:: b_{x,y}^{i} = {a_{x,y}^{i}}{({k + {}{n} {j=max(0,
where the sum runs over :math:n
"adjacent" kernel maps at the same spatial number of kernels in the layer.
Defined in src/operator/nn/lrn.cc:L164
symbol_name | name of the resulting symbol |
data | Input data to LRN |
nsize | normalization window width in elements. |
alpha | The variance scaling parameter :math:lpha in the LRN expression. |
beta | The power parameter :math:eta in the LRN expression. |
knorm | The parameter :math:k in the LRN expression. |
|
inline |
Applies local response normalization to the input.
The local response normalization layer performs "lateral inhibition" by over local input regions.
If :math:a_{x,y}^{i}
is the activity of a neuron computed by applying kernel :math:(x, y)
and then applying the ReLU nonlinearity, the response-normalized activity :math:b_{x,y}^{i}
is given by the expression:
.. math:: b_{x,y}^{i} = {a_{x,y}^{i}}{({k + {}{n} {j=max(0,
where the sum runs over :math:n
"adjacent" kernel maps at the same spatial number of kernels in the layer.
Defined in src/operator/nn/lrn.cc:L164
data | Input data to LRN |
nsize | normalization window width in elements. |
alpha | The variance scaling parameter :math:lpha in the LRN expression. |
beta | The power parameter :math:eta in the LRN expression. |
knorm | The parameter :math:k in the LRN expression. |
|
inline |
Computes mean absolute error of the input.
MAE is a risk metric corresponding to the expected value of the absolute error.
If :math:\hat{y}_i
is the predicted value of the i-th sample, and :math:y_i
then the mean absolute error (MAE) estimated over :math:n
samples is defined
:math:`{MAE}({Y}, {{Y}} ) = {1}{n} {i=0}^{n-1}
.. note:: Use the MAERegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
By default, gradients of this loss function are scaled by factor 1/m
, where m The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L120
symbol_name | name of the resulting symbol |
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
|
inline |
Computes mean absolute error of the input.
MAE is a risk metric corresponding to the expected value of the absolute error.
If :math:\hat{y}_i
is the predicted value of the i-th sample, and :math:y_i
then the mean absolute error (MAE) estimated over :math:n
samples is defined
:math:`{MAE}({Y}, {{Y}} ) = {1}{n} {i=0}^{n-1}
.. note:: Use the MAERegressionOutput as the final output layer of a net.
The storage type of label
can be default
or csr
By default, gradients of this loss function are scaled by factor 1/m
, where m The parameter grad_scale
can be used to change this scale to grad_scale/m
.
Defined in src/operator/regression_output.cc:L120
data | Input data to the function. |
label | Input label to the function. |
grad_scale | Scale the gradient by a float factor |
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input
For example, if you are a making a cross entropy loss function. Assume out
predicted output and label
is the true label, then the cross entropy can be
cross_entropy = label * log(out) + (1 - label) * log(1 - out) loss = make_loss(cross_entropy)
We will need to use make_loss
when we are creating our own loss function or combine multiple loss functions. Also we may want to stop some variables' from backpropagation. See more detail in BlockGrad
or stop_gradient
.
The storage type of make_loss
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L332
symbol_name | name of the resulting symbol |
data | The input array. |
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input
For example, if you are a making a cross entropy loss function. Assume out
predicted output and label
is the true label, then the cross entropy can be
cross_entropy = label * log(out) + (1 - label) * log(1 - out) loss = make_loss(cross_entropy)
We will need to use make_loss
when we are creating our own loss function or combine multiple loss functions. Also we may want to stop some variables' from backpropagation. See more detail in BlockGrad
or stop_gradient
.
The storage type of make_loss
output depends upon the input storage type:
Defined in src/operator/tensor/elemwise_unary_op_basic.cc:L332
data | The input array. |
|
inline |
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input
For example, if you are a making a cross entropy loss function. Assume out
predicted output and label
is the true label, then the cross entropy can be
cross_entropy = label * log(out) + (1 - label) * log(1 - out) loss = MakeLoss(cross_entropy)
We will need to use MakeLoss
when we are creating our own loss function or combine multiple loss functions. Also we may want to stop some variables' from backpropagation. See more detail in BlockGrad
or stop_gradient
.
In addition, we can give a scale to the loss by setting grad_scale
, so that the gradient of the loss will be rescaled in the backpropagation.
.. note:: This operator should be used as a Symbol instead of NDArray.
Defined in src/operator/make_loss.cc:L71
symbol_name | name of the resulting symbol |
data | Input array. |
grad_scale | Gradient scale as a supplement to unary and binary operators |
valid_thresh | clip each element in the array to 0 when it is less than |
normalization | If this is set to null, the output gradient will not be normalized. If this is set to batch, the output gradient will be divided by the batch size. If this is set to valid, the output gradient will be divided by the |
|
inline |
Make your own loss function in network construction.
This operator accepts a customized loss function symbol as a terminal loss and the symbol should be an operator with no backward dependency. The output of this function is the gradient of loss with respect to the input
For example, if you are a making a cross entropy loss function. Assume out
predicted output and label
is the true label, then the cross entropy can be
cross_entropy = label * log(out) + (1 - label) * log(1 - out) loss = MakeLoss(cross_entropy)
We will need to use MakeLoss
when we are creating our own loss function or combine multiple loss functions. Also we may want to stop some variables' from backpropagation. See more detail in BlockGrad
or stop_gradient
.
In addition, we can give a scale to the loss by setting grad_scale
, so that the gradient of the loss will be rescaled in the backpropagation.
.. note:: This operator should be used as a Symbol instead of NDArray.
Defined in src/operator/make_loss.cc:L71
data | Input array. |
grad_scale | Gradient scale as a supplement to unary and binary operators |
valid_thresh | clip each element in the array to 0 when it is less than |
normalization | If this is set to null, the output gradient will not be normalized. If this is set to batch, the output gradient will be divided by the batch size. If this is set to valid, the output gradient will be divided by the |
|
inline |
Computes the max of array elements over given axes.
Defined in src/operator/tensor/broadcast_reduce_op_value.cc:L191
symbol_name | name of the resulting symbol |
data | The input |
axis | The axis or axes along which to perform the reduction. The default, `axis=()`, will compute over all elements into a scalar array with shape `(1,)`. If `axis` is int, a reduction is performed on a particular axis. If `axis` is a tuple of ints, a reduction is performed on all the axes specified in the tuple. If `exclude` is true, reduction will be performed on |