Linear Algebra Symbol API¶
Overview¶
This document lists the linear algebra routines of the symbolic expression package:
mxnet.symbol.linalg 
Linear Algebra Symbol API of MXNet. 
The Linear Algebra Symbol
API, defined in the symbol.linalg
package, provides
symbolic expressions for linear algebra routines.
In the rest of this document, we list routines provided by the symbol.linalg
package.
Linear Algebra¶
gemm 
Performs general matrix multiplication and accumulation. 
gemm2 
Performs general matrix multiplication. 
potrf 
Performs Cholesky factorization of a symmetric positivedefinite matrix. 
potri 
Performs matrix inversion from a Cholesky factorization. 
trmm 
Performs multiplication with a lower triangular matrix. 
trsm 
Solves matrix equation involving a lower triangular matrix. 
sumlogdiag 
Computes the sum of the logarithms of the diagonal elements of a square matrix. 
syrk 
Multiplication of matrix with its transpose. 
gelqf 
LQ factorization for general matrix. 
syevd 
Eigendecomposition for symmetric matrix. 
API Reference¶
Linear Algebra Symbol API of MXNet.

mxnet.symbol.linalg.
gelqf
(A=None, name=None, attr=None, out=None, **kwargs)¶ LQ factorization for general matrix. Input is a tensor A of dimension n >= 2.
If n=2, we compute the LQ factorization (LAPACK gelqf, followed by orglq). A must have shape (x, y) with x <= y, and must have full rank =x. The LQ factorization consists of L with shape (x, x) and Q with shape (x, y), so that:
A = L * QHere, L is lower triangular (upper triangle equal to zero) with nonzero diagonal, and Q is roworthonormal, meaning that
Q * Q^{T}is equal to the identity matrix of shape (x, x).
If n>2, gelqf is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single LQ factorization A = [[1., 2., 3.], [4., 5., 6.]] Q, L = gelqf(A) Q = [[0.26726124, 0.53452248, 0.80178373], [0.87287156, 0.21821789, 0.43643578]] L = [[3.74165739, 0.], [8.55235974, 1.96396101]] // Batch LQ factorization A = [[[1., 2., 3.], [4., 5., 6.]], [[7., 8., 9.], [10., 11., 12.]]] Q, L = gelqf(A) Q = [[[0.26726124, 0.53452248, 0.80178373], [0.87287156, 0.21821789, 0.43643578]], [[0.50257071, 0.57436653, 0.64616234], [0.7620735, 0.05862104, 0.64483142]]] L = [[[3.74165739, 0.], [8.55235974, 1.96396101]], [[13.92838828, 0.], [19.09768702, 0.52758934]]]
Defined in src/operator/tensor/la_op.cc:L569
Parameters:  A (Symbol) – Tensor of input matrices to be factorized
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
gemm
(A=None, B=None, C=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, beta=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)¶ Performs general matrix multiplication and accumulation. Input are tensors A, B, C, each of dimension n >= 2 and having the same shape on the leading n2 dimensions.
If n=2, the BLAS3 function gemm is performed:
out = alpha * op(A) * op(B) + beta * CHere, alpha and beta are scalar parameters, and op() is either the identity or matrix transposition (depending on transpose_a, transpose_b).
If n>2, gemm is performed separately for a batch of matrices. The column indices of the matrices are given by the last dimensions of the tensors, the row indices by the axis specified with the axis parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a nondefault axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes calls. For example let A, B, C be 5 dimensional tensors. Then gemm(A, B, C, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3) B1 = swapaxes(B, dim1=1, dim2=3) C = swapaxes(C, dim1=1, dim2=3) C = gemm(A1, B1, C) C = swapaxis(C, dim1=1, dim2=3)without the overhead of the additional swapaxis operations.
When the input data is of type float32 and the environment variables MXNET_CUDA_ALLOW_TENSOR_CORE and MXNET_CUDA_TENSOR_OP_MATH_ALLOW_CONVERSION are set to 1, this operator will try to use pseudofloat16 precision (float32 math with float16 I/O) precision in order to use Tensor Cores on suitable NVIDIA GPUs. This can sometimes give significant speedups.
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiplyadd A = [[1.0, 1.0], [1.0, 1.0]] B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]] C = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] gemm(A, B, C, transpose_b=True, alpha=2.0, beta=10.0) = [[14.0, 14.0, 14.0], [14.0, 14.0, 14.0]] // Batch matrix multiplyadd A = [[[1.0, 1.0]], [[0.1, 0.1]]] B = [[[1.0, 1.0]], [[0.1, 0.1]]] C = [[[10.0]], [[0.01]]] gemm(A, B, C, transpose_b=True, alpha=2.0 , beta=10.0) = [[[104.0]], [[0.14]]]
Defined in src/operator/tensor/la_op.cc:L87
Parameters:  A (Symbol) – Tensor of input matrices
 B (Symbol) – Tensor of input matrices
 C (Symbol) – Tensor of input matrices
 transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A).
 transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B).
 alpha (double, optional, default=1) – Scalar factor multiplied with A*B.
 beta (double, optional, default=1) – Scalar factor multiplied with C.
 axis (int, optional, default='2') – Axis corresponding to the matrix rows.
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
gemm2
(A=None, B=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, axis=_Null, name=None, attr=None, out=None, **kwargs)¶ Performs general matrix multiplication. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n2 dimensions.
If n=2, the BLAS3 function gemm is performed:
out = alpha * op(A) * op(B)Here alpha is a scalar parameter and op() is either the identity or the matrix transposition (depending on transpose_a, transpose_b).
If n>2, gemm is performed separately for a batch of matrices. The column indices of the matrices are given by the last dimensions of the tensors, the row indices by the axis specified with the axis parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a nondefault axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes calls. For example let A, B be 5 dimensional tensors. Then gemm(A, B, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3) B1 = swapaxes(B, dim1=1, dim2=3) C = gemm2(A1, B1) C = swapaxis(C, dim1=1, dim2=3)without the overhead of the additional swapaxis operations.
When the input data is of type float32 and the environment variables MXNET_CUDA_ALLOW_TENSOR_CORE and MXNET_CUDA_TENSOR_OP_MATH_ALLOW_CONVERSION are set to 1, this operator will try to use pseudofloat16 precision (float32 math with float16 I/O) precision in order to use Tensor Cores on suitable NVIDIA GPUs. This can sometimes give significant speedups.
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiply A = [[1.0, 1.0], [1.0, 1.0]] B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]] gemm2(A, B, transpose_b=True, alpha=2.0) = [[4.0, 4.0, 4.0], [4.0, 4.0, 4.0]] // Batch matrix multiply A = [[[1.0, 1.0]], [[0.1, 0.1]]] B = [[[1.0, 1.0]], [[0.1, 0.1]]] gemm2(A, B, transpose_b=True, alpha=2.0) = [[[4.0]], [[0.04 ]]]
Defined in src/operator/tensor/la_op.cc:L162
Parameters:  A (Symbol) – Tensor of input matrices
 B (Symbol) – Tensor of input matrices
 transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A).
 transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B).
 alpha (double, optional, default=1) – Scalar factor multiplied with A*B.
 axis (int, optional, default='2') – Axis corresponding to the matrix row indices.
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
potrf
(A=None, name=None, attr=None, out=None, **kwargs)¶ Performs Cholesky factorization of a symmetric positivedefinite matrix. Input is a tensor A of dimension n >= 2.
If n=2, the Cholesky factor B of the symmetric, positive definite matrix A is computed. B is triangular (entries of upper or lower triangle are all zero), has positive diagonal entries, and:
A = B * B^{T} if lower = true A = B^{T} * B if lower = falseIf n>2, potrf is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix factorization A = [[4.0, 1.0], [1.0, 4.25]] potrf(A) = [[2.0, 0], [0.5, 2.0]] // Batch matrix factorization A = [[[4.0, 1.0], [1.0, 4.25]], [[16.0, 4.0], [4.0, 17.0]]] potrf(A) = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]]
Defined in src/operator/tensor/la_op.cc:L213
Parameters:  A (Symbol) – Tensor of input matrices to be decomposed
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
potri
(A=None, name=None, attr=None, out=None, **kwargs)¶ Performs matrix inversion from a Cholesky factorization. Input is a tensor A of dimension n >= 2.
If n=2, A is a triangular matrix (entries of upper or lower triangle are all zero) with positive diagonal. We compute:
out = A^{T} * A^{1} if lower = true out = A^{1} * A^{T} if lower = falseIn other words, if A is the Cholesky factor of a symmetric positive definite matrix B (obtained by potrf), then
out = B^{1}If n>2, potri is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Note
Use this operator only if you are certain you need the inverse of B, and cannot use the Cholesky factor A (potrf), together with backsubstitution (trsm). The latter is numerically much safer, and also cheaper.
Examples:
// Single matrix inverse A = [[2.0, 0], [0.5, 2.0]] potri(A) = [[0.26563, 0.0625], [0.0625, 0.25]] // Batch matrix inverse A = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]] potri(A) = [[[0.26563, 0.0625], [0.0625, 0.25]], [[0.06641, 0.01562], [0.01562, 0,0625]]]
Defined in src/operator/tensor/la_op.cc:L274
Parameters:  A (Symbol) – Tensor of lower triangular matrices
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
sumlogdiag
(A=None, name=None, attr=None, out=None, **kwargs)¶ Computes the sum of the logarithms of the diagonal elements of a square matrix. Input is a tensor A of dimension n >= 2.
If n=2, A must be square with positive diagonal entries. We sum the natural logarithms of the diagonal elements, the result has shape (1,).
If n>2, sumlogdiag is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix reduction A = [[1.0, 1.0], [1.0, 7.0]] sumlogdiag(A) = [1.9459] // Batch matrix reduction A = [[[1.0, 1.0], [1.0, 7.0]], [[3.0, 0], [0, 17.0]]] sumlogdiag(A) = [1.9459, 3.9318]
Defined in src/operator/tensor/la_op.cc:L445
Parameters:  A (Symbol) – Tensor of square matrices
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
syevd
(A=None, name=None, attr=None, out=None, **kwargs)¶ Eigendecomposition for symmetric matrix. Input is a tensor A of dimension n >= 2.
If n=2, A must be symmetric, of shape (x, x). We compute the eigendecomposition, resulting in the orthonormal matrix U of eigenvectors, shape (x, x), and the vector L of eigenvalues, shape (x,), so that:
U * A = diag(L) * UHere:
U * U^{T} = U^{T} * U = Iwhere I is the identity matrix. Also, L(0) <= L(1) <= L(2) <= ... (ascending order).
If n>2, syevd is performed separately on the trailing two dimensions of A (batch mode). In this case, U has n dimensions like A, and L has n1 dimensions.
Note
The operator supports float32 and float64 data types only.
Note
Derivatives for this operator are defined only if A is such that all its eigenvalues are distinct, and the eigengaps are not too small. If you need gradients, do not apply this operator to matrices with multiple eigenvalues.
Examples:
// Single symmetric eigendecomposition A = [[1., 2.], [2., 4.]] U, L = syevd(A) U = [[0.89442719, 0.4472136], [0.4472136, 0.89442719]] L = [0., 5.] // Batch symmetric eigendecomposition A = [[[1., 2.], [2., 4.]], [[1., 2.], [2., 5.]]] U, L = syevd(A) U = [[[0.89442719, 0.4472136], [0.4472136, 0.89442719]], [[0.92387953, 0.38268343], [0.38268343, 0.92387953]]] L = [[0., 5.], [0.17157288, 5.82842712]]
Defined in src/operator/tensor/la_op.cc:L638
Parameters:  A (Symbol) – Tensor of input matrices to be factorized
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
syrk
(A=None, transpose=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)¶ Multiplication of matrix with its transpose. Input is a tensor A of dimension n >= 2.
If n=2, the operator performs the BLAS3 function syrk:
out = alpha * A * A^{T}if transpose=False, or
out = alpha * A^{T} * Aif transpose=True.
If n>2, syrk is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiply A = [[1., 2., 3.], [4., 5., 6.]] syrk(A, alpha=1., transpose=False) = [[14., 32.], [32., 77.]] syrk(A, alpha=1., transpose=True) = [[17., 22., 27.], [22., 29., 36.], [27., 36., 45.]] // Batch matrix multiply A = [[[1., 1.]], [[0.1, 0.1]]] syrk(A, alpha=2., transpose=False) = [[[4.]], [[0.04]]]
Defined in src/operator/tensor/la_op.cc:L501
Parameters:  A (Symbol) – Tensor of input matrices
 transpose (boolean, optional, default=0) – Use transpose of input matrix.
 alpha (double, optional, default=1) – Scalar factor to be applied to the result.
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
trmm
(A=None, B=None, transpose=_Null, rightside=_Null, lower=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)¶ Performs multiplication with a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n2 dimensions.
If n=2, A must be triangular. The operator performs the BLAS3 function trmm:
out = alpha * op(A) * Bif rightside=False, or
out = alpha * B * op(A)if rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).
If n>2, trmm is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single triangular matrix multiply A = [[1.0, 0], [1.0, 1.0]] B = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] trmm(A, B, alpha=2.0) = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]] // Batch triangular matrix multiply A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]] B = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]], [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]]] trmm(A, B, alpha=2.0) = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]], [[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]]
Defined in src/operator/tensor/la_op.cc:L333
Parameters:  A (Symbol) – Tensor of lower triangular matrices
 B (Symbol) – Tensor of matrices
 transpose (boolean, optional, default=0) – Use transposed of the triangular matrix
 rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to nontriangular one.
 lower (boolean, optional, default=1) – True if the triangular matrix is lower triangular, false if it is upper triangular.
 alpha (double, optional, default=1) – Scalar factor to be applied to the result.
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type:

mxnet.symbol.linalg.
trsm
(A=None, B=None, transpose=_Null, rightside=_Null, lower=_Null, alpha=_Null, name=None, attr=None, out=None, **kwargs)¶ Solves matrix equation involving a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n2 dimensions.
If n=2, A must be triangular. The operator performs the BLAS3 function trsm, solving for out in:
op(A) * out = alpha * Bif rightside=False, or
out * op(A) = alpha * Bif rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).
If n>2, trsm is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix solve A = [[1.0, 0], [1.0, 1.0]] B = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]] trsm(A, B, alpha=0.5) = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] // Batch matrix solve A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]] B = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]], [[4.0, 4.0, 4.0], [8.0, 8.0, 8.0]]] trsm(A, B, alpha=0.5) = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]], [[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]
Defined in src/operator/tensor/la_op.cc:L396
Parameters:  A (Symbol) – Tensor of lower triangular matrices
 B (Symbol) – Tensor of matrices
 transpose (boolean, optional, default=0) – Use transposed of the triangular matrix
 rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to nontriangular one.
 lower (boolean, optional, default=1) – True if the triangular matrix is lower triangular, false if it is upper triangular.
 alpha (double, optional, default=1) – Scalar factor to be applied to the result.
 name (string, optional.) – Name of the resulting symbol.
Returns: The result symbol.
Return type: