\[
{\bf A} = \left[ a_{i,j} \right] = \left[ a_{i,j} \right]_{i,j=1}^{m,n} \qquad \mbox{and} \qquad {\bf B} = \left( b_{i,j} \right) = \left( b_{i,j} \right)_{i,j=1}^{n} ,
\]
being a square matrix. The notation (A)
is also common, depending on the setting.
The subscripts denote the position of the entry. The entry 𝑎i,j or just 𝑎ij occupies the i-th row and j-th column of the matrix A. Two matrices A and B are equal if they have the same dimensions and 𝑎ij = bij for every i and j.
We define four arithmetic operations on matrices: Matrix addition or subtraction, scalar multiplication, and matrix multiplication. Matrix division is considered in the next section.
\[
\begin{bmatrix} \phantom{-}2&7 \\ -1&3 \\ -4&6
\end{bmatrix} + \begin{bmatrix} 3&-4 \\ 2& -8 \\ 3&-3
\end{bmatrix} = \begin{bmatrix} \phantom{-}2+3 &7-4 \\ -1+2 & 3 -8 \\ -4 + 3 & 6-3
\end{bmatrix} = \begin{bmatrix} \phantom{-}5&\phantom{-}3 \\ \phantom{-}1&-5 \\ -1&\phantom{-}3
\end{bmatrix}
\]
\[
\begin{pmatrix} 2&-1&-4 \\ 7&\phantom{-}3&\phantom{-}6
\end{pmatrix} - \begin{pmatrix} \phantom{-}3&\phantom{-}2&\phantom{-}3 \\ -4&-8&-3
\end{pmatrix} = \begin{pmatrix} -1&-1&-7 \\ 11&11&\phantom{-}9
\end{pmatrix} .
\]
To be more formal---and to begin to get used to the abstract notation---we could express this idea as
\[
{\bf A} \pm {\bf B} = \left[ a_{i,j} \right] \pm \left[ b_{i,j} \right] = \left[ a_{i,j} \pm b_{i,j} \right] .
\]
\[
k\, {\bf A} = k \left[ a_{i,j} \right] = \left[ k\,a_{i,j} \right] , \qquad k\in \mathbb{R}.
\]
We don't know exactly who invented nor when the multiplication of matrices was invented. However, we do know that the work of 1812 by Jacques Philippe Marie Binet (1786--1856) contains the definition of the product of matrices. Let A [𝑎i,j] be a m × n matrix and B = [bi,j] be a n × k matrix. Its product, a \( {\bf C} = {\bf A}\,{\bf B} \) is a
m × k matrix, in which the n entries across the rows of A are multiplied with the n entries down the columns of B:
\[
{\bf C} = \left[ c_{ij} \right] , \quad\mbox{where} \quad c_{ij}= \sum_{k=1}^n a_{ik}b_{kj} .
\]
of the second factor. Their product is denoted by
\begin{equation} \label{EqBasic.1}
{\bf C} = \left[ c_{i,j} \right] = {\bf A}. {\bf B} , \qquad c_{i,j} = \sum_{k=1}^m a_{i,k} b_{k,j} , \quad i = 1, 2, \ldots , m; \quad j=1,2,\ldots , n.
\end{equation}
\[
{\bf A} = \left[ a_{i,j} \right] \qquad \mbox{and} \qquad {\bf B} = \left[ b_{i,j} \right] ,
\]
\[
{\bf A} \ast {\bf B} = \left[ a_{i,j}* b_{i,j} \right] \qquad \mbox{and} \qquad {\bf A} / {\bf B} = \left[ a_{i,j} / b_{i,j} \right] = \left[ \frac{a_{i,j}}{b_{i,j}} \right] .
\]
\[
{\bf A} = \begin{bmatrix} 1&4&3 \\ 2&-1&2 \\ 1&2&2 \end{bmatrix} \qquad \Longrightarrow \qquad {\bf A}\ast {\bf A} = \begin{bmatrix} 1&16&9 \\ 4&1&4 \\ 1 & 4 & 4 \end{bmatrix} \qquad \mbox{and} \qquad {\bf A} / {\bf A} = \begin{bmatrix} 1&1&1 \\ 1&1&1 \\ 1&1&1 \end{bmatrix} .
\]
Now we start working with matrices. A period “.” can also be used for matrix multiplication between one matrix and a vector or two matrices. It is important to note that when doing matrix multiplication an (m x n) matrix can only be multiplied by an (n x s) where m, n, and s are whole numbers, producing an (m x s) matrix.
matlab knows what vector should be used when a matrix is multiplied by a vector. For example,
A={{16,-9},{-12,13}};
v={1,2};
A.v
Out[3]= {-2,14}
v.A
Out[4]= {-8, 17}
This example shows that when a matrix is multiplied by a vector from the right (this also means that a matrix is operated on a vector as a transformation),
matlab treats it as a column-vector. When the vector is multiplied by a matrix from the right,
matlab treats the same vector as a
row-vector.
However, we can specify either row-vector or column-vector and multiply by a
matrix from left or right:
v={2,1,-1} (* row-vector *)
u = {{2}, {1}, {-1}} (* column-vector *)
Products:
A.v
Out[17]= {1, 1} (* row *)
A.u
Out[18]= {{1}, {1}} (* column *)
Now we generate a special matrix, called the zero matrix, whose entries are
all zeroes.
Z23 = Table[0, {i, 2}, {j, 3}] (* generates a list of zero values *)
Out[1]= {{0, 0, 0}, {0, 0, 0}}
This matrix can be added (or subtracted from) to any matrix of the same dimensions (in our case, it is 2 × 3 matrix). We can verify this by showing that
A +
Z23 is equal to
A:
A + Z23 == A
Out[2]= True
A == M (* when A ≠ M *)
Out[6]= False
Example 1:
Write the vector
\( {\bf a}=2{\bf i}+3{\bf j}-4{\bf k} \) as the sum of two vectors, one parallel, and one
perpendicular to the vector
\( {\bf b}=2{\bf i}-{\bf j}-3{\bf k} .\)
We find the parallel vector:
a = {2, 3, -4}
b = {2, -1, -3}
madB2 = (b[[1]])^2 + (b[[2]])^2 + (b[[3]])^2
Aparallel = a.b/madB2*b
Out[4]= {13/7, -(13/14), -(39/14)}
To find the perpendicular vector, we type:
Aperpendicular = a - Aparallel
Out[5]= {1/7, 55/14, -(17/14)}
Another more straight forward way:
Aparallel = (a.b)*b/(b.b)
To verify the answer, we type:
Aparallel + Aperpendicular -a
Out[9]= 0
Example 2:
Let
a = [1,3,-4] and
b = [-1,1,-2] be two vectors. Find
\( {\bf a}\cdot ({\bf a}\times {\bf b}), \)
where
\( {\bf a}\cdot {\bf b} \) is dot product, and
\( {\bf a}\times {\bf b} \) is cross product of two vectors. Input variables are vectors
a and
b. Function will calculate the cross product of
a and
b, and then
calculate the dot product of that value and
a. Notice that any input values for
a and
b will
result in zero sum.
a={1,3,-4}
b={-1,1,-2}
c=a.Cross[a,b]
{0,0,0}
{-1,1,-2}
0
As a simple introductory example, consider 2×2 matrix of zero trace:
\[
{\bf A} = \begin{bmatrix} \phantom{-}0 & 1 \\ -1& 0 \end{bmatrix} .
\]
Now we check with
matlab:
(A = {{0, 1}, {-1, 0}}) // MatrixForm
Out[2]= \( \begin{pmatrix} 0 & 1 \\ -1&0 \end{pmatrix} \)
Tr[A]
Out[2]= 0
As you see,
matlab has a dedicated command to evaluate the trace:
Tr[·], which we will abreviate as tr(·).
The set ℳm,n of all m × n matrices under the field of either real or complex numbers is a vector space of dimension m · n.
In order to determine how close two matrices are, and in order to define the convergence of sequences of matrices, a special concept of matrix norm is employed, with notation \( \| {\bf A} \| . \) A norm
is a function from a real or complex vector space to the nonnegative real numbers that satisfies the following conditions:
-
Positivity: ‖A‖ ≥ 0
‖A‖ = 0 iff A = 0.
-
Homogeneity: ‖kA‖ = |k| ‖A‖ for arbitrary scalar k.
-
Triangle inequality: ‖A + B‖ ≤ ‖A‖ + ‖B‖.
Once a norm is defined, it is the most natural way of measure distance between two matrices
A and
B as d(
A,
B) = ‖
A −
B‖ = ‖
B −
A‖. However, not all distance functions have a corresponding norm. For example, a trivial distance that has no equivalent norm is d(
A,
A) = 0 and d(
A,
B) = 1 if
A ≠
B.
The norm of a matrix may be thought of as its magnitude or length because it is a nonnegative number.
Their definitions are summarized below for an
\( m \times n \) matrix
A, to which corresponds a
self-adjoint (
m+
n)×(
m+
n) matrix
B:
\[
{\bf A} = \left[ \begin{array}{cccc} a_{1,1} & a_{1,2} & \cdots & a_{1,n} \\
a_{2,1} & a_{2,2} & \cdots & a_{2,n} \\
\vdots & \vdots & \ddots & \vdots \\
a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{array} \right] \qquad \Longrightarrow \qquad {\bf B} = \begin{bmatrix} {\bf 0} & {\bf A}^{\ast} \\ {\bf A} & {\bf 0} \end{bmatrix} .
\]
Here
A* denotes the adjoint matrix:
\( {\bf A}^{\ast} = \overline{{\bf A}^{\mathrm T}} = \overline{\bf A}^{\mathrm T} . \)
For a rectangular
m-by-
n matrix
A and a given norms
\( \| \ \| \)
in
\( \mathbb{R}^n \mbox{ and } \mathbb{R}^m , \) the
norm of
A is defined as follows:
\begin{equation} \label{EqBasic.2}
\| {\bf A} \| = \sup_{{\bf x} \ne {\bf 0}} \ \dfrac{\| {\bf A}\,{\bf x} \|_m}{\| {\bf x} \|_n} = \sup_{\| {\bf x} \| = 1} \ \| {\bf A}\,{\bf x} \| .
\end{equation}
This matrix norm is called the
operator norm or induced norm.
The term "induced" refers to the fact that the definition of a norm for vectors such as
A x and
x is what enables the definition above of a matrix norm.
This definition of matrix norm is not computationally friendly, so we use other options. The most important norms are as follow
✼
The operator norm corresponding to the p-norm for vectors, p ≥ 1, is:
\begin{equation} \label{EqBasic.3}
\| {\bf A} \|_p = \sup_{{\bf x} \ne 0} \, \frac{\| {\bf A}\,{\bf x} \|_p}{\| {\bf x} \|_p} = \sup_{\| {\bf x} \|_p =1} \, \| {\bf A}\,{\bf x} \|_p ,
\end{equation}
where
\( \| {\bf x} \|_p = \left( x_1^p + x_2^p + \cdots + x_n^p \right)^{1/p} .\)
✼
1-norm (is commonly known as the maximum column sum norm) of a matrix A may be computed as
\begin{equation} \label{EqBasic.4}
\| {\bf A} \|_1 = \max_{1 \le j \le n} \,\sum_{i=1}^n | a_{i,j} | .
\end{equation}
✻
The infinity norm, \( \infty - \) norm of matrix A may be computed as
\begin{equation} \label{EqBasic.5}
\| {\bf A} \|_{\infty} = \| {\bf A}^{\ast} \|_{1} = \max_{1 \le i \le n} \,\sum_{j=1}^n | a_{i,j} | ,
\end{equation}
which is simply the maximum absolute row sum of the matrix.
In
matlab this norm is implemented by the function norm(A,inf)
✺
In the special case of p = 2 we get the Euclidean norm (which is equal to the largest singular value of a matrix)
\begin{equation} \label{EqBasic.6}
\| {\bf A} \|_2 = \sup_{\bf x} \left\{ \| {\bf A}\, {\bf x} \|_2 \, : \quad \mbox{with} \quad \| {\bf x} \|_2 =1 \right\} = \sigma_{\max} \left( {\bf A} \right) = \sqrt{\rho \left( {\bf A}^{\ast} {\bf A} \right)} ,
\end{equation}
where σ
max(
A) represents the largest
singular value of matrix
A.
✳
The Frobenius norm (not induced norm):
\begin{equation} \label{EqBasic.7}
\| {\bf A} \|_F = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{i.j} |^2 \right)^{1/2} = \left( \mbox{tr}\, {\bf A} \,{\bf A}^{\ast} \right)^{1/2} = \left( \mbox{tr}\, {\bf A}^{\ast} {\bf A} \right)^{1/2} .
\end{equation}
The Euclidean norm and the Frobenius norm are related via the inequality:
\[
\| {\bf A} \|_2 = \sigma_{\max}\left( {\bf A} \right) \le \| {\bf A} \|_F = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{i.j} |^2 \right)^{1/2} = \left( \mbox{tr}\, {\bf A} \,{\bf A}^{\ast} \right)^{1/2} .
\]
There is also another function that that provides infinum of all norms of a squar matrix: \( \rho ({\bf A}) \le \|{\bf A}\| . \)
The
spectral radius of a square matrix
A is
\begin{equation} \label{EqBasic.8}
\rho ({\bf A}) = \lim_{k\to \infty} \| {\bf A}^k \|^{1/k} = \max \left\{ |\lambda | : \ \lambda \mbox{ is eigenvalue of }\ {\bf A} \right\} .
\end{equation}
For any positive integer
k, we have
\begin{equation} \label{EqBasic.9}
\rho ({\bf A}) \le \| {\bf A}^k \|^{1/k} .
\end{equation}
Some properties of the matrix norms are presented in the following
Theorem 1:
Let
A and
B be
\( m \times n \) matrices
and let
\( k \) be a scalar.
- \( \| {\bf A} \| \ge 0 \) for any square matrix A.
- \( \| {\bf A} \| =0 \) if and only if the matrix A is zero: \( {\bf A} = {\bf 0}. \)
- \( \| k\,{\bf A} \| = |k| \, \| {\bf A} \| \) for any scalar \( k. \)
- \( \| {\bf A} + {\bf B}\| \le \| {\bf A} \| + \| {\bf B} \| .\)
- \( \left\vert \| {\bf A} - {\bf B}\|\right\vert \le \| {\bf A} - {\bf B} \| .\)
- \( \| {\bf A} \, {\bf B}\| \le \| {\bf A} \| \, \| {\bf B} \| . \)
All these norms are equivalent and we present some inequalities:
\[
\| {\bf A} \|_2^2 \le \| {\bf A}^{\ast} \|_{\infty} \cdot \| {\bf A} \|_{\infty} = \| {\bf A} \|_{1} \cdot \| {\bf A} \|_{\infty} ,
\]
where
A* is the adjoint matrix to
A (transposed and complex conjugate).
Theorem 2:
Let ‖ ‖ be any matrix norm and let
B be a matrix such that
B < 1. If ‖ ‖ is any operator norm, then matrix
I +
B is invertible and
\[
\| \left( {\bf I} + {\bf B} \right)^{-1} \| \le \frac{1}{1 - \| {\bf B} \|} .
\]
Theorem 3:
Let ‖ ‖ be any matrix norm, and let matrix
I + B is singular, where
I is the identity matrix.
Then B ≥ 1 for every matrix norm.
matlab
has a special command for evaluating norms:
norm(A) = norm(A,2) for evaluating the Euclidean norm of the matrix A;
norm(A, 1) for evaluating the 1-norm;
norm(A, inf) for evaluating the ∞-norm;
norm(A, 'fro') for evaluating the Frobenius norm;
A = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}
Norm[A]
Sqrt[3/2 (95 + Sqrt[8881])]
N[%]
16.8481
Example 3:
Evaluate the norms of the matrix
\( {\bf A} = \left[ \begin{array}{cc} \phantom{-}1 & -7 & 4 \\ -2 & -3 & 1\end{array} \right] . \)
The absolute column sums of A are \( 1 + | -2 | =3 \) , \( |-7| + | -3 | =10 , \) and \( 4+1 =5 . \)
The larger of these is 10 and therefore \( \| {\bf A} \|_1 = 10 . \)
Norm[A, 1]
10
The absolute row sums of A are \( 1 + | -7 | + 4 =12 \) and
\( | -2 | + |-3| + 1 = 6 ; \) therefore, \( \| {\bf A} \|_{\infty} = 12 . \)
Norm[Transpose[A], 1]
12
The Euclidean norm of A
is the largest singular value. So we calculate
\[
{\bf S} = {\bf A}^{\ast} {\bf A} = \begin{bmatrix} 5&-1&2 \\ -1&58&-31 \\ 2&-31&17 \end{bmatrix} , \qquad \mbox{tr} \left( {\bf S} \right) = 80.
\]
Its eigenvalues are
Eigenvalues[Transpose[A].A]
{40 + Sqrt[1205], 40 - Sqrt[1205], 0}
Taking the square root of the largest one, we obtain the Euclidean norm of matrix
A:
N[Sqrt[40 + Sqrt[1205]]]
8.64367
matlab also knows how to find the Euclidean norm:
Norm[A, 2]
Sqrt[40 + Sqrt[1205]]
We compare it with the Frobenius norm:
Norm[A, "Frobenius"]
4 Sqrt[5]
N[%]
8.94427
Norm[A]
Sqrt[40 + Sqrt[1205]]
N[%]
8.64367
To find its exact value, we evaluate the product
\[
{\bf M} = {\bf A}\,{\bf A}^{\ast} = \left[ \begin{array}{cc} \phantom{-}1 & -7 & 4\\ -2 & -3 & 1 \end{array} \right] \,
\left[ \begin{array}{cc} 1 & -2 \\ -7 & -3 \\ 4&-1 \end{array} \right] =
\left[ \begin{array}{cc} 66 & 23 \\ 23& 14 \end{array} \right] , \qquad \mbox{tr} \left( {\bf M} \right) = 80.
\]
This matrix\( {\bf A}\,{\bf A}^{\ast} \) has two eigenvalues
\( 40 \pm \sqrt{1205} . \) Hence, the Euclidean norm of the matrix A is
\( \sqrt{40 + \sqrt{1205}} \approx 8.64367 . \)
Therefore,
\[
\| {\bf A} \|_2 = 8.64367 < \| {\bf A} \|_F = 8.94427 < \| {\bf A} \|_1 = 10 < \| {\bf A} \|_{\infty} = 12 .
\]
■
Example 4:
Let us consider the matrix
\[
{\bf A} = \begin{bmatrix} 1&2&3 \\ 4&5&6 \\ 7&8&9 \end{bmatrix} .
\]
Its conjugate transpose (adjoint) matrix is
\[
{\bf A}^{\ast} = \begin{bmatrix} 1&2&3 \\ 4&5&6 \\ 7&8&9 \end{bmatrix}^{\mathrm T} = \begin{bmatrix} 1&4&7 \\ 2&5&8 \\ 3&6&9 \end{bmatrix} .
\]
So
\[
{\bf S} = {\bf A}^{\ast} {\bf A} = \begin{bmatrix} 66&78&90 \\ 78&93&108 \\ 90&108&126 \end{bmatrix}
\]
We check with
matlab:
A = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}
S = Transpose[A].A
Their eigenvalues are
Eigenvalues[A]
{3/2 (5 + Sqrt[33]), 3/2 (5 - Sqrt[33]), 0}
Eigenvalues[S]
{3/2 (95 + Sqrt[8881]), 3/2 (95 - Sqrt[8881]), 0}
N[%]
{283.859, 1.14141, 0.}
Therefore, the largest singular number of
A is
\[
\sigma = \sqrt{\frac{3}{2} \left( 95 + \sqrt{8881} \right)} \approx 16.8481.
\]
We also check the opposite product
Eigenvalues[A.Transpose[A]]
{3/2 (95 + Sqrt[8881]), 3/2 (95 - Sqrt[8881]), 0}
\[
{\bf M} = {\bf A}\, {\bf A}^{\ast} = \begin{bmatrix} 14&32&50 \\ 32&77&122 \\ 50&122&194 \end{bmatrix}
\]
These matrices
S and
M have the same eigenvalues. Therefore, we found the Euclidean (operator) norm of
A to be approximately 16.8481.
matlab knows this norm:
Norm[A]
Sqrt[3/2 (95 + Sqrt[8881])]
The spectral radius of
A is the largest eigenvalue:
\[
\rho ({\bf A}) = \frac{3}{2} \left( 5 + \sqrt{33} \right) \approx 16.1168 ,
\]
which is slightly less than its operator (Euclidean) norm.
The Frobenius norm of matrix \( {\bf A} = \begin{bmatrix} 1&2&3 \\ 4&5&6 \\ 7&8&9 \end{bmatrix} \) is
\[
\| {\bf A} \|_F = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{i.j} |^2 \right)^{1/2} = \left( 1^2 + 2^2 + 3^2 + 4^2 + 5^2 + 6^2 + 7^2 +8^2 +9^2 \right)^{1/2} = \sqrt{285} = \left( \mbox{tr}\, {\bf A} \,{\bf A}^{\ast} \right)^{1/2} .
\]
A = {{1, 2, 3}, {4, 5, 6}, {7, 8, 9}}
Tr[A.Transpose[A]]
285
Sum[k^2, {k, 1, 9}]
285
N[Sqrt[285]]
16.8819
matlab has a dedicated command:
Norm[A, "Frobenius"]
16.8819
To find 1-norm of A, we add elements in every column; it turns out that the last column has the largest entries, so
\[
\| {\bf A} \|_1 = 3+6+9=18.
\]
If we add entries in every row, then the last row contains the lagest values and we get
\[
\| {\bf A} \|_{\infty} = 7+8+9=24.
\]
■
Since vectors are a special case of matrices, the definitions of norms for matrices can be extended for vectors. For every ,i>n-vector
x = [
x1,
x2,…,
xn], we have
-
1-norm
\[
\| {\bf x} \|_{1} = | x_1 | + |x_2 | + \cdots + |x_n | .
\]
-
the Chebyshev norm
\[
\| {\bf x} \|_{\infty} = \max_{1 \le i \le n} \left\{ | x_i | \right\}
\]
-
the Euclidean norm
\[
\| {\bf x} \|_{2} = \left( |x_1 |^2 + |x_2 |^2 + \cdots + |x_n |^2 \right)^{1/2} .
\]
-
p-norm
\[
\| {\bf x} \|_{p} = \left( |x_1 |^p + |x_2 |^p + \cdots + |x_n |^p \right)^{1/p} .
\]
To find the Euclidean length of a vector use the Norm[vector] operation. The command norm( )
also works for finding the norm of a complex number, where the imaginary unit is represented by “I.”
Norm[{5,2,9,7,3}]
2 Sqrt[42]
This definition can be extended to matrices:
Norm[{{0, 5}, {-1, 6}}]
Out[6]= Sqrt[31 + 6 Sqrt[26]]
For any two vectors
u and
v of the same dimensions, we have
Hölder's inequality
\[
\sum_{i=1}^n \left\vert v_i u_i \right\vert \le \| {\bf v} \|_p \| {\bf u} \|_q , \qquad \frac{1}{p} + \frac{1}{q} = 1.
\]
Theorem 4:
The following inequalities hold for all
x ∈ ℝn or
x ∈ ℂn
\begin{align*}
&\| {\bf x} \|_{\infty} \le \| {\bf x} \|_{1} \le n\,\| {\bf x} \|_{\infty} ,
\\
&\| {\bf x} \|_{\infty} \le \| {\bf x} \|_{2} \le \sqrt{n}\,\| {\bf x} \|_{\infty} ,
\\
&\| {\bf x} \|_{2} \le \| {\bf x} \|_{1} \le \sqrt{n}\, \| {\bf x} \|_{2} .
\end{align*}
======================================================
Try addition and multiplication for starters. Observe what happens:
A = [1 2; 3 4]
B = [5 6; 7 8]
AplusB = A + B
AtimesB = A * B
AtimesB is
A times
B, which is the result of standard matrix multiplication from linear
algebra. However, often one is interested in the element by element
multiplication of two matrices. The way to do this in
matlab is using the
.* operator, where the dot indicated element by element calculation. Try
it, and compare the results:
Other operations that we have discussed in class include calculating the
transpose, the complex conjugate and the adjoint of a matrix. The
transpose of a real-valued matrix
is simply
while for a matrix
that also contains some complex entries
calculates the complex conjugate.
To calculate the transpose of such a matrix, one has to use the familiar .:
Theorem: If the sizes of the matrices are such that the stated operations can be performed, then:
- \( \left({\bf A}^T \right)^T = {\bf A} \) for any matrix A;
- \( \left( {\bf A} + {\bf B} \right)^T = {\bf A}^T + {\bf B}^T .\)
- \( \left( {\bf A} - {\bf B} \right)^T = {\bf A}^T - {\bf B}^T .\)
- \( \left( k\,{\bf A} \right)^T = k\, {\bf A}^T .\)
- \( \left( {\bf A} \, {\bf B}\right)^T = {\bf B}^T \, {\bf A}^T . \)
The norm of a matrix may be thought of as its size because it is a nonnegative number. Matrix norms are directly related to vector norms.
The definitions are summarized below for an \( m \times n \) matrix A.
\[
{\bf A} = \left[ \begin{array}{cccc} a_{1,1} & a_{1,2} & \cdots & a_{1,n} \\
a_{2,1} & a_{2,2} & \cdots & a_{2,n} \\
\vdots & \vdots & \ddots & \vdots \\
a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{array} \right] .
\]
The operator norm corresponding to the p-norm for vectors, p ≥ 1, is:
\[
\| {\bf A} \|_p = \sup_{{\bf x} \ne 0} \, \frac{\| {\bf A}\,{\bf x} \|_p}{\| {\bf x} \|_p} ,
\]
where
\( \| {\bf x} \|_p = \left( x_1^p + x_2^p + \cdots + x_n^p \right)^{1/p} .\)
The most important norms are
1-norm (is commonly known as the maximum column sum norm) of a matrix A may be computed as
\[
\| {\bf A} \|_1 = \max_{1 \le j \le n} \,\sum_{i=1}^n | a_{i,j} | .
\]
The infinity norm,
\( \infty - \) norm of matrix
A may be computed as
\[
\| {\bf A} \|_{\infty} = \max_{1 \le i \le n} \,\sum_{j=1}^n | a_{i,j} | ,
\]
which is simply the maximum absolute row sum of the matrix.
In the special case of
p = 2 we get the Euclidean norm (which is equal to the largest singular value of a matrix)
\[
\| {\bf A} \|_2 = \sup_{\bf x} \left\{ \| {\bf A}\, {\bf x} \|_2 \, : \quad \mbox{with} \quad \| {\bf x} \|_2 =1 \right\} .
\]
The Frobenius norm:
\[
\| {\bf A} \|_F = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{i.j} |^2 \right)^{1/2} = \left( \mbox{tr}\, {\bf A} \,{\bf A}^{\ast} \right)^{1/2} .
\]
Some properties of the matrix norms are presented in the following
Theorem 1:
Let
A and
B be
\( m \times n \) matrices
and let
\( k \) be a scalar.
- \( \| {\bf A} \| \ge 0 \) for any square matrix A.
- \( \| {\bf A} \| =0 \) if and only if the matrix A is zero: \( {\bf A} = {\bf 0}. \)
- \( \| k\,{\bf A} \| = |k| \, \| {\bf A} \| \) for any scalar \( k. \)
- \( \| {\bf A} + {\bf B}\| \le \| {\bf A} \| + \| {\bf B} \| .\)
- \( \| {\bf A} \, {\bf B}\| \le \| {\bf A} \| \, \| {\bf B} \| \)
matlab has special commands for evaluating norms:
norm(A,1) for evaluating 1-norm of the matrix A;
norm(A,inf) for evaluating \( infty - \)norm of the matrix A;
norm(A,2) for evaluating 2-norm (Euclidean) of the matrix A;
Example:
Evaluate the norms of the matrix
\( {\bf A} = \left[ \begin{array}{cc} 1 & -7 \\ -2 & -3 \end{array} \right] . \)
The absolute column sums of A are \( 1 + | -2 | =3 \) and \( |-7| + | -3 | =10 . \)
The larger of these is 10 and therefore \( \| {\bf A} \|_1 = 10 . \)
The absolute row sums of A are \( 1 + | -7 | =8 \) and
\( | -2 | + |-3| = 5 , \) therefore, \( \| {\bf A} \|_{\infty} = 8 . \)
The Euclidean norm of A
is
\[
\| {\bf A} \|_2 = \sup_{\bf x} \left\{ \, \sqrt{(x_1 - 7\,x_2 )^2 + (2\,x_1 + 3\,x_2 )^2} \, : \quad \mbox{with} \quad x_1^2 + x_2^2 =1 \right\} .
\]
To find its exact value, we evaluate the product
\[
{\bf A}\,{\bf A}^{\ast} = \left[ \begin{array}{cc} 1 & -7 \\ -2 & -3 \end{array} \right] \,
\left[ \begin{array}{cc} 1 & -2 \\ -7 & -3 \end{array} \right] =
\left[ \begin{array}{cc} 50 & 19 \\ 19 & 13 \end{array} \right] .
\]
This matrix\( {\bf A}\,{\bf A}^{\ast} \) has two eigenvalues
\( \frac{1}{2} \left( 63 \pm \sqrt{2813} \right) . \) Hence, the Euclidean norm of the matrix A is
\( \sqrt{\frac{1}{2} \left( 63 + \sqrt{2813} \right)} . \)
■
The Frobenius norm:
\[
\| {\bf A} \|_F = \left( \sum_{i=1}^m \sum_{j=1}^n |a_{i.j} |^2 \right)^{1/2} = \left( 1+49+4+9 \right)^{1/2} = \sqrt{63} = \left( \mbox{tr}\, {\bf A} \,{\bf A}^{\ast} \right)^{1/2} .
\]