Linear Algebra NDArray API¶
Overview¶
This document lists the linear algebra routines of the n-dimensional array package:
mxnet.ndarray.linalg |
Linear Algebra NDArray API of MXNet. |
The Linear Algebra NDArray
API, defined in the ndarray.linalg
package, provides
imperative linear algebra tensor operations on CPU/GPU.
In the rest of this document, we list routines provided by the ndarray.linalg
package.
Linear Algebra¶
gemm |
Performs general matrix multiplication and accumulation. |
gemm2 |
Performs general matrix multiplication. |
potrf |
Performs Cholesky factorization of a symmetric positive-definite matrix. |
potri |
Performs matrix inversion from a Cholesky factorization. |
trmm |
Performs multiplication with a lower triangular matrix. |
trsm |
Solves matrix equation involving a lower triangular matrix. |
sumlogdiag |
Computes the sum of the logarithms of the diagonal elements of a square matrix. |
syrk |
Multiplication of matrix with its transpose. |
gelqf |
LQ factorization for general matrix. |
syevd |
Eigendecomposition for symmetric matrix. |
API Reference¶
Linear Algebra NDArray API of MXNet.
-
mxnet.ndarray.linalg.
gelqf
(A=None, out=None, name=None, **kwargs)¶ LQ factorization for general matrix. Input is a tensor A of dimension n >= 2.
If n=2, we compute the LQ factorization (LAPACK gelqf, followed by orglq). A must have shape (x, y) with x <= y, and must have full rank =x. The LQ factorization consists of L with shape (x, x) and Q with shape (x, y), so that:
A = L * QHere, L is lower triangular (upper triangle equal to zero) with nonzero diagonal, and Q is row-orthonormal, meaning that
Q * QTis equal to the identity matrix of shape (x, x).
If n>2, gelqf is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single LQ factorization A = [[1., 2., 3.], [4., 5., 6.]] Q, L = gelqf(A) Q = [[-0.26726124, -0.53452248, -0.80178373], [0.87287156, 0.21821789, -0.43643578]] L = [[-3.74165739, 0.], [-8.55235974, 1.96396101]] // Batch LQ factorization A = [[[1., 2., 3.], [4., 5., 6.]], [[7., 8., 9.], [10., 11., 12.]]] Q, L = gelqf(A) Q = [[[-0.26726124, -0.53452248, -0.80178373], [0.87287156, 0.21821789, -0.43643578]], [[-0.50257071, -0.57436653, -0.64616234], [0.7620735, 0.05862104, -0.64483142]]] L = [[[-3.74165739, 0.], [-8.55235974, 1.96396101]], [[-13.92838828, 0.], [-19.09768702, 0.52758934]]]
Defined in src/operator/tensor/la_op.cc:L552
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
gemm
(A=None, B=None, C=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, beta=_Null, axis=_Null, out=None, name=None, **kwargs)¶ Performs general matrix multiplication and accumulation. Input are tensors A, B, C, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.
If n=2, the BLAS3 function gemm is performed:
out = alpha * op(A) * op(B) + beta * CHere, alpha and beta are scalar parameters, and op() is either the identity or matrix transposition (depending on transpose_a, transpose_b).
If n>2, gemm is performed separately for a batch of matrices. The column indices of the matrices are given by the last dimensions of the tensors, the row indices by the axis specified with the axis parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a non-default axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes calls. For example let A, B, C be 5 dimensional tensors. Then gemm(A, B, C, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3) B1 = swapaxes(B, dim1=1, dim2=3) C = swapaxes(C, dim1=1, dim2=3) C = gemm(A1, B1, C) C = swapaxis(C, dim1=1, dim2=3)without the overhead of the additional swapaxis operations.
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiply-add A = [[1.0, 1.0], [1.0, 1.0]] B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]] C = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] gemm(A, B, C, transpose_b=True, alpha=2.0, beta=10.0) = [[14.0, 14.0, 14.0], [14.0, 14.0, 14.0]] // Batch matrix multiply-add A = [[[1.0, 1.0]], [[0.1, 0.1]]] B = [[[1.0, 1.0]], [[0.1, 0.1]]] C = [[[10.0]], [[0.01]]] gemm(A, B, C, transpose_b=True, alpha=2.0 , beta=10.0) = [[[104.0]], [[0.14]]]
Defined in src/operator/tensor/la_op.cc:L81
Parameters: - A (NDArray) – Tensor of input matrices
- B (NDArray) – Tensor of input matrices
- C (NDArray) – Tensor of input matrices
- transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A).
- transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B).
- alpha (double, optional, default=1) – Scalar factor multiplied with A*B.
- beta (double, optional, default=1) – Scalar factor multiplied with C.
- axis (int, optional, default='-2') – Axis corresponding to the matrix rows.
- out (NDArray, optional) – The output NDArray to hold the result.
Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
gemm2
(A=None, B=None, transpose_a=_Null, transpose_b=_Null, alpha=_Null, axis=_Null, out=None, name=None, **kwargs)¶ Performs general matrix multiplication. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.
If n=2, the BLAS3 function gemm is performed:
out = alpha * op(A) * op(B)Here alpha is a scalar parameter and op() is either the identity or the matrix transposition (depending on transpose_a, transpose_b).
If n>2, gemm is performed separately for a batch of matrices. The column indices of the matrices are given by the last dimensions of the tensors, the row indices by the axis specified with the axis parameter. By default, the trailing two dimensions will be used for matrix encoding.
For a non-default axis parameter, the operation performed is equivalent to a series of swapaxes/gemm/swapaxes calls. For example let A, B be 5 dimensional tensors. Then gemm(A, B, axis=1) is equivalent to
A1 = swapaxes(A, dim1=1, dim2=3) B1 = swapaxes(B, dim1=1, dim2=3) C = gemm2(A1, B1) C = swapaxis(C, dim1=1, dim2=3)without the overhead of the additional swapaxis operations.
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiply A = [[1.0, 1.0], [1.0, 1.0]] B = [[1.0, 1.0], [1.0, 1.0], [1.0, 1.0]] gemm2(A, B, transpose_b=True, alpha=2.0) = [[4.0, 4.0, 4.0], [4.0, 4.0, 4.0]] // Batch matrix multiply A = [[[1.0, 1.0]], [[0.1, 0.1]]] B = [[[1.0, 1.0]], [[0.1, 0.1]]] gemm2(A, B, transpose_b=True, alpha=2.0) = [[[4.0]], [[0.04 ]]]
Defined in src/operator/tensor/la_op.cc:L151
Parameters: - A (NDArray) – Tensor of input matrices
- B (NDArray) – Tensor of input matrices
- transpose_a (boolean, optional, default=0) – Multiply with transposed of first input (A).
- transpose_b (boolean, optional, default=0) – Multiply with transposed of second input (B).
- alpha (double, optional, default=1) – Scalar factor multiplied with A*B.
- axis (int, optional, default='-2') – Axis corresponding to the matrix row indices.
- out (NDArray, optional) – The output NDArray to hold the result.
Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
potrf
(A=None, out=None, name=None, **kwargs)¶ Performs Cholesky factorization of a symmetric positive-definite matrix. Input is a tensor A of dimension n >= 2.
If n=2, the Cholesky factor L of the symmetric, positive definite matrix A is computed. L is lower triangular (entries of upper triangle are all zero), has positive diagonal entries, and:
A = L * LTIf n>2, potrf is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix factorization A = [[4.0, 1.0], [1.0, 4.25]] potrf(A) = [[2.0, 0], [0.5, 2.0]] // Batch matrix factorization A = [[[4.0, 1.0], [1.0, 4.25]], [[16.0, 4.0], [4.0, 17.0]]] potrf(A) = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]]
Defined in src/operator/tensor/la_op.cc:L201
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
potri
(A=None, out=None, name=None, **kwargs)¶ Performs matrix inversion from a Cholesky factorization. Input is a tensor A of dimension n >= 2.
If n=2, A is a lower triangular matrix (entries of upper triangle are all zero) with positive diagonal. We compute:
out = A-T * A-1In other words, if A is the Cholesky factor of a symmetric positive definite matrix B (obtained by potrf), then
out = B-1If n>2, potri is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Note
Use this operator only if you are certain you need the inverse of B, and cannot use the Cholesky factor A (potrf), together with backsubstitution (trsm). The latter is numerically much safer, and also cheaper.
Examples:
// Single matrix inverse A = [[2.0, 0], [0.5, 2.0]] potri(A) = [[0.26563, -0.0625], [-0.0625, 0.25]] // Batch matrix inverse A = [[[2.0, 0], [0.5, 2.0]], [[4.0, 0], [1.0, 4.0]]] potri(A) = [[[0.26563, -0.0625], [-0.0625, 0.25]], [[0.06641, -0.01562], [-0.01562, 0,0625]]]
Defined in src/operator/tensor/la_op.cc:L259
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
sumlogdiag
(A=None, out=None, name=None, **kwargs)¶ Computes the sum of the logarithms of the diagonal elements of a square matrix. Input is a tensor A of dimension n >= 2.
If n=2, A must be square with positive diagonal entries. We sum the natural logarithms of the diagonal elements, the result has shape (1,).
If n>2, sumlogdiag is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix reduction A = [[1.0, 1.0], [1.0, 7.0]] sumlogdiag(A) = [1.9459] // Batch matrix reduction A = [[[1.0, 1.0], [1.0, 7.0]], [[3.0, 0], [0, 17.0]]] sumlogdiag(A) = [1.9459, 3.9318]
Defined in src/operator/tensor/la_op.cc:L428
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
syevd
(A=None, out=None, name=None, **kwargs)¶ Eigendecomposition for symmetric matrix. Input is a tensor A of dimension n >= 2.
If n=2, A must be symmetric, of shape (x, x). We compute the eigendecomposition, resulting in the orthonormal matrix U of eigenvectors, shape (x, x), and the vector L of eigenvalues, shape (x,), so that:
U * A = diag(L) * UHere:
U * UT = UT * U = Iwhere I is the identity matrix. Also, L(0) <= L(1) <= L(2) <= ... (ascending order).
If n>2, syevd is performed separately on the trailing two dimensions of A (batch mode). In this case, U has n dimensions like A, and L has n-1 dimensions.
Note
The operator supports float32 and float64 data types only.
Note
Derivatives for this operator are defined only if A is such that all its eigenvalues are distinct, and the eigengaps are not too small. If you need gradients, do not apply this operator to matrices with multiple eigenvalues.
Examples:
// Single symmetric eigendecomposition A = [[1., 2.], [2., 4.]] U, L = syevd(A) U = [[0.89442719, -0.4472136], [0.4472136, 0.89442719]] L = [0., 5.] // Batch symmetric eigendecomposition A = [[[1., 2.], [2., 4.]], [[1., 2.], [2., 5.]]] U, L = syevd(A) U = [[[0.89442719, -0.4472136], [0.4472136, 0.89442719]], [[0.92387953, -0.38268343], [0.38268343, 0.92387953]]] L = [[0., 5.], [0.17157288, 5.82842712]]
Defined in src/operator/tensor/la_op.cc:L621
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
syrk
(A=None, transpose=_Null, alpha=_Null, out=None, name=None, **kwargs)¶ Multiplication of matrix with its transpose. Input is a tensor A of dimension n >= 2.
If n=2, the operator performs the BLAS3 function syrk:
out = alpha * A * ATif transpose=False, or
out = alpha * AT * Aif transpose=True.
If n>2, syrk is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix multiply A = [[1., 2., 3.], [4., 5., 6.]] syrk(A, alpha=1., transpose=False) = [[14., 32.], [32., 77.]] syrk(A, alpha=1., transpose=True) = [[17., 22., 27.], [22., 29., 36.], [27., 36., 45.]] // Batch matrix multiply A = [[[1., 1.]], [[0.1, 0.1]]] syrk(A, alpha=2., transpose=False) = [[[4.]], [[0.04]]]
Defined in src/operator/tensor/la_op.cc:L484
Parameters: Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
trmm
(A=None, B=None, transpose=_Null, rightside=_Null, alpha=_Null, out=None, name=None, **kwargs)¶ Performs multiplication with a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.
If n=2, A must be lower triangular. The operator performs the BLAS3 function trmm:
out = alpha * op(A) * Bif rightside=False, or
out = alpha * B * op(A)if rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).
If n>2, trmm is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single triangular matrix multiply A = [[1.0, 0], [1.0, 1.0]] B = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] trmm(A, B, alpha=2.0) = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]] // Batch triangular matrix multiply A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]] B = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]], [[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]]] trmm(A, B, alpha=2.0) = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]], [[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]]
Defined in src/operator/tensor/la_op.cc:L316
Parameters: - A (NDArray) – Tensor of lower triangular matrices
- B (NDArray) – Tensor of matrices
- transpose (boolean, optional, default=0) – Use transposed of the triangular matrix
- rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to non-triangular one.
- alpha (double, optional, default=1) – Scalar factor to be applied to the result.
- out (NDArray, optional) – The output NDArray to hold the result.
Returns: out – The output of this function.
Return type: NDArray or list of NDArrays
-
mxnet.ndarray.linalg.
trsm
(A=None, B=None, transpose=_Null, rightside=_Null, alpha=_Null, out=None, name=None, **kwargs)¶ Solves matrix equation involving a lower triangular matrix. Input are tensors A, B, each of dimension n >= 2 and having the same shape on the leading n-2 dimensions.
If n=2, A must be lower triangular. The operator performs the BLAS3 function trsm, solving for out in:
op(A) * out = alpha * Bif rightside=False, or
out * op(A) = alpha * Bif rightside=True. Here, alpha is a scalar parameter, and op() is either the identity or the matrix transposition (depending on transpose).
If n>2, trsm is performed separately on the trailing two dimensions for all inputs (batch mode).
Note
The operator supports float32 and float64 data types only.
Examples:
// Single matrix solve A = [[1.0, 0], [1.0, 1.0]] B = [[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]] trsm(A, B, alpha=0.5) = [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]] // Batch matrix solve A = [[[1.0, 0], [1.0, 1.0]], [[1.0, 0], [1.0, 1.0]]] B = [[[2.0, 2.0, 2.0], [4.0, 4.0, 4.0]], [[4.0, 4.0, 4.0], [8.0, 8.0, 8.0]]] trsm(A, B, alpha=0.5) = [[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]], [[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]]
Defined in src/operator/tensor/la_op.cc:L379
Parameters: - A (NDArray) – Tensor of lower triangular matrices
- B (NDArray) – Tensor of matrices
- transpose (boolean, optional, default=0) – Use transposed of the triangular matrix
- rightside (boolean, optional, default=0) – Multiply triangular matrix from the right to non-triangular one.
- alpha (double, optional, default=1) – Scalar factor to be applied to the result.
- out (NDArray, optional) – The output NDArray to hold the result.
Returns: out – The output of this function.
Return type: NDArray or list of NDArrays