Matrix Norms

The absolute column sums of A are \( 1 + | -2 | =3 \) , \( |-7| + | -3 | =10 , \) and \( 4+1 =5 . \) The larger of these is 10 and therefore \( \| {\bf A} \|_1 = 10 . \)

The absolute row sums of A are \( 1 + | -7 | + 4 =12 \) and \( | -2 | + |-3| + 1 = 6 ; \) therefore, \( \| {\bf A} \|_{\infty} = 12 . \)

The Euclidean norm of A is the largest singular value. So we calculate

This matrix\( {\bf A}\,{\bf A}^{\ast} \) has two eigenvalues \( 40 \pm \sqrt{1205} . \) Hence, the Euclidean norm of the matrix A is \( \sqrt{40 + \sqrt{1205}} \approx 8.64367 . \)

The Frobenius norm of matrix \( {\bf A} = \begin{bmatrix} 1&2&3 \\ 4&5&6 \\ 7&8&9 \end{bmatrix} \) is

To find 1-norm of A, we add elements in every column; it turns out that the last column has the largest entries, so

The set ℳ_m,n of all m × n matrices under the field of either real or complex numbers is a vector space of dimension m  · n. In order to determine how close two matrices are, or to define the convergence of sequences of matrices, a special concept of matrix norm is employed, with notation \( \| {\bf A} \| . \) A norm is a function from a real or complex vector space to the nonnegative real numbers that satisfies the following conditions:

• Positivity:     ‖A‖ ≥ 0,     ‖A‖ = 0 iff A = 0.
• Homogeneity:     ‖kA‖ = |k| ‖A‖ for arbitrary scalar k.
• Triangle inequality:     ‖A + B‖ ≤ ‖A‖ + ‖B‖.

• The operator norms are norms induced by a matrix considered as a linear operator from ℝ^m into ℝⁿ for real scalars or ℂ^m into ℂⁿ for complex scalars.
• The entrywise norms treat an m-by-n matrix as the vector of length m · n. Therefore, these norms are directly related to norms in a vector space.
• The Schatten norms are based on the singular values σ_i or eigenvalues of the given matrix.

The term "induced" refers to the fact that the definition of a norm for vectors such as A x and x is what enables the definition above of a matrix norm. This definition of matrix norm is not computationally friendly, so we use other options. The following popular norms are listed below.

• Norm[A, 1] for evaluating ‖ · ‖₁;
• Norm[A] = Norm[A, 2] for evaluating the Euclidean norm;
• Norm[A, Infinity] for evaluating ‖ · ‖_∞.

The induced matrix norms constitute a large and important part of possible matrix norms---there are known many non-induced norms. The following very important non-induced norm is called after Ferdinand Georg Frobenius (1849--1917).

One can think of the Frobenius norm as taking the columns of the matrix, stacking them on top of each other to create a vector of size m×n, and then taking the vector 2-norm of the result. The Frobenius norm is the matrix norm that is unitary invariant, i.e., it is conserved or invariant under a unitary transformation (such as a rotation). For a norm to be unitary invariant, it should depend solely upon the singular values of the matrix. So if B = R*A R with a unitary (orthogonal if real) matrix R satisfying R* = R^-1, then

There is also another function that that provides infinum of all norms of a square matrix: \( \rho ({\bf A}) \le \|{\bf A}\| . \)

• \( \| {\bf A} \|_1 \) is the absolute sum of all elements of A:
  \begin{equation} \label{EqMatrix.7a} \| {\bf A} \|_1 = \sum_{i=1}^m \,\sum_{j=1}^n |a_{ij} | . \end{equation}
• \( \| {\bf A} \|_{\max} \) is the maximum norm, the maximum absolute value among all mn elements of A:
  \begin{equation} \label{EqMatrix.9a} \| {\bf A} \|_{\max} = \max_{i,j} \,|a_{ij} | . \end{equation}

Robert Schatten (1911--1977) suggested to define a matrix norm based on the singular values σ_i or eigenvalues. Therefore, these norms are called after him. We present three of them, with overlapping previously used notations. We denote by r the rank of the rectangular m×n matrix A.

• \( \| {\bf A} \|_1 \) is the trace norm:
  \[ \| {\bf A} \|_1 = \sum_{i=1}^r \sigma_i = \sum_{i=1}^r \sqrt{\lambda_i} \left[ = \mbox{trace} \left( \sqrt{{\bf A}^{\ast} {\bf A}} \right) \right] . \]
The last formula is for disposition because a square root of a matrix may not exit, but when it exists, it is not unique.
• \( \| {\bf A} \|_2 = \| {\bf A} \|_F \) is the Frobenius norm.
\[ \| {\bf A} \|_2 = \sqrt{ \sum_{i=1}^r \sigma_i^2 } = \sqrt{ \sum_{i=1}^r \lambda_i } = \| {\bf A} \|_F . \]
• \( \| {\bf A} \|_{\infty} \) is the spectral norm (the spectral radius of A*A):
  \[ \| {\bf A} \|_{\infty} = \max \left\{ \sigma_1 , \ldots , \sigma_n \right\} = \sigma_{\max} \left( {\bf A} \right) = \sqrt{\lambda_{\max} \left( {\bf A}^{\ast} {\bf A} \right)} . \]