es

The Wolfram Mathematica notebook which contains the code that produces all the Mathematica output in this web page may be downloaded at this link.

We denote by 𝔽 a field of numbers of which we consider only four types: ℤ, a set of integers, ℚ, a set of rational numbers, ℝ, a set of real numbers, and ℂ, a set of complex numbers.

Dot Product

We met many times in previous sections a special linear combination of numerical vectors. For instance, a linear equation in n unknowns
\[ a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = b , \tag{0} \]
which we prefer to write in succinct form a • x = b, where a = (𝑎₁, 𝑎₂, … , 𝑎n), x = (x₁, x₂, … , xn), and b = (b₁, b₂, … , bn) are numerical vectors from 𝔽n.
Let α = [ a₁, a₂, … , an ] and β = [ b₁, b₂, … , bn ] be two ordered orthonormal bases of vector spaces V and U over the same field 𝔽, respectively. Then any two vectors from these vector spaces are uniquely represented by n-tuples. The dot product or scalar product of two vectors of the same size \( {\bf x} = \left[ x_1 , x_2 , \ldots , x_n \right] \) and \( {\bf y} = \left[ y_1 , y_2 , \ldots , y_n \right] \) is the number from field 𝔽, denoted by x • y, \begin{equation} \label{EqDot.1} {\bf x} \bullet {\bf y} = x_1 y_1 + x_2 y_2 + \cdots + x_n y_n . \end{equation}
Remark:    The algebraic formula (1) uses definition of inner product and orthonormality, which are the topics of the following sections. Existence of orthonormal basis is essential for our presentation; if you are not familiar with orthogonality, you can use standard rectangular coordinates and Cartesian product ℝn.

Geometric interpretation of the dot product, which is coordinate independent and therefore conveys invariant properties of these products, is given in the Euclidean space section. The dot product is not defined for vectors of different dimensions. It does not matter whether vectors are columns or rows or n-tuples. so you can evaluate dot product of row vectors with column vectors---they must be from the vector spaces over the same field. Therefore, this definition is valid not only for n-tuples (elements from 𝔽n), but also for column vectors and row vectors. Mathematica does not distinguish rows from columns. Dot product can be accomplished with two Mathematica commands:

a = {1, 2, 3}; b = {3, 2, 1};
Dot[a, b]
a . b
Josiah Gibbs
The dot product was first introduced by the American physicist and mathematician Josiah Willard Gibbs (1839--1903) in the 1880s.

Many years before Gibbs definition, ancient Greeks discovered that geometrically the product of the corresponding entries of the two sequences of numbers is equivalent to the product of their magnitudes and the cosine of the angle between them. This leads to introducing a etric (or length or distance) in the Cartesian product ℝ³ transferring it into the Euclidean space. Originally, it was the three-dimensional physical space, but in modern mathematics there are Euclidean spaces of any positive integer dimension n, which are called Euclidean n-spaces.

At the beginning of twentieth century, it was discovered that the dot product is needed for definition of dual spaces (see section in Part3). Then left-hand part of Eq.(1) defines a linear functional on any n dimensional vector space independently on what field is used (ℂ or ℝ). Then one of the multipliers, say x in Eq.(1), is called a vector, but another counterpart, y is known as a covector. Such treatment of vectors in the dot product breaks their equal rights; in many practical problems, these vectors, x and y, are indeed different, but sometimes look the same.

In geometry, to distinguish these two partners in Eq. (1), the vector x is called contravariant vector, and the covector y is referred to as covariablt vector. In order to decide between these partners who is who, it is common to use supersript for coordinates of contravariant vector, x = [ x¹, x², x³], and subscript for covariant vectors, y = [y₁, y₂, y₃]. In physics, covariant vectors are also called bra-vectors, while contravariant vectors are known as ket-vectors.

However, a vector space, by definition, has no metric inside it, which is very desirable property. It turns out that the scalar product can be used to define length or distance between vectors transfering ℝn iinto a metric space, known as the Euclidean space. In 1912, the Hungarian mathematician Frigyes Riesz established an isomorphism between an Euclidean space and its dual space. His result (which is also valid for some infinite dimensional spaces) restores equal right between vectors and covectors, but under a new marriage sectificate---known as the inner product, which is our next topic to discuss.

The following basic properties of the dot product are important. They are all easily proven from the above definition. In the following properties, u, , and w are n-dimensional vectors, and λ is a number (scalar):

  • u • v = v • u       (commutative law)
  • (u + v) • w = u • w + v • w;       (distributive law)
  • (λ u) • v = λ (u • v).
   
Example 1:    ■
End of Example 1

 

Euclidean Space

The invention of Cartesian coordinates in 1649 by René Descartes (Latinized name: Cartesius) revolutionized mathematics by providing the first systematic link between Euclidean geometry and algebra.

   
Example 2:    ■
End of Example 2

 

Geometric Properties of the Dot Product

Geometrical analysis yields further interesting properties of the dot product operation that can then be used in nongeometric applications. This takes a little work.

Consider a fixed two-dimensional coordinate system with origin at point O. Let P = (px, py) and Q = (qx, qy) be two arbitrary points on the plane ℝ². When Euclidean norm ‖·‖2 is employed, we can define the distance from the origin to any point on the plane. For example, the distance from the origin to point P is

\[ \left\| \vec{OP} \right\| = \sqrt{p_x^2 + p_y^2} . \]

 

Dot Product and Linear Transformations

The fundamental significance of a dot product is that it is a linear transformtion of vectors. This means that the function f(v) = u • v is a linear functional for any fixed vector u. The following famous theorem (proved independently by Frigyes Riesz and Maurice René Fréchet in 1907) establishes the converse: Any linear functional T(v) corresponds to the dot product with a weight vector u.
Riesz representation theorem: Let f be a linear form on n-dimensional vector space V. Then there exists a unique vector u such that for all v ∈ V, f(v) = u • v

 

Application of the Dot Product: Weighted Sum

The dot product is very important in physics. Let us consider an example. In classical mechanics it is true that the ‘work’ that is done when an object is moved equals the dot product of the force acting on the object and the displacement vector:

\[ F = \mathbf{F} \bullet \mathbf{x} . \]
The work W must of course be independent of the coordinate system in which the vectors F and x are expressed. The dot product as we know it from Eq.(1) does not have this property. In general, using matrix transformation, we have
\[ s = {\bf A}\,\mathbf{x} \bullet {\bf A}\,\mathbf{y} = {\bf A}^{\mathrm T} {\bf A}\,\mathbf{x} \bullet \mathbf{y} . \]
Only if A−1 equals AT (i.e. if we are dealing with orthonormal transformations) s will not change. It appears as if the dot product only describes the physics correctly in a special kind of coordinate system: a system which according to our human perception is ‘rectangular’, and has physical units, i.e. a distance of 1 in coordinate x means indeed 1 meter in x-direction. An orthonormal transformation produces again such a rectangular ‘physical’ coordinate system. If one has so far always employed such special coordinates anyway, this dot product has always worked properly.

It is not always guaranteed that one can use such special coordinate systems (polar coordinates are an example in which the local orthonormal basis of vectors is not the coordinate basis). However, the dot product between a vector x and a covector y is invariant under all transformations because this product defines a functional generated by covector y. Then the given dot product is just one representation of this linear functional in particular coordinates. Making linear transformation with matrix A, we get

\begin{align*} \mathbf{x} \bullet \mathbf{y} &= \sum_i x^i y_i = \sum_i \sum_j A^i_j \xi^j \bullet \left( \mathbf{A}^{-1} \right)^j_i \eta_j \\ &= \sum_j \sum_i \left( \mathbf{A}^{-1} \right)^j_i A^i_j \xi^j \bullet \eta_j = \sum_j \xi^j \bullet \eta_j . \end{align*}

We can use the dot product to find the angle between two vectors. From the definition of the dot product, we get

\[ {\bf a} \cdot {\bf b} = \langle {\bf a} , {\bf b} \rangle = \| {\bf a} \| \cdot \| {\bf b} \| \,\cos \theta , \]
where θ is the angle between ywo vectors a and b. If the vectors are nonzero, then
\[ \theta = \arccos \left( \frac{{\bf a} \cdot {\bf b}}{\| {\bf a} \| \cdot \| {\bf b} \| } \right) . \]

The prime example of dot operation is work that is defined as the scalar product of force and displacement. The presence of cos(θ) ensures the requirement that the work done by a force perpendicular to the displacement is zero.

The dot product is clearly commutative, 𝑎 · b = b · 𝑎. Moreover, it distributes over vector addition

\[ ({\bf a} + {\bf b}) · {\bf c} = {\bf a} · {\bf c} + {\bf b} · {\bf c}. \]

One can use the distributive property of the dot product to show that if (ax, ay, az) and (bx, by, bz) represent the components of a and b along the axes x, y, and z, then

\[ {\bf a} \cdot {\bf b} = a_x b_x + a_y b_y + a_z b_z . \]
From the definition of the dot product, we can draw an important conclusion. If we divide both sides of a · b = |a| |b| cos θ by |a|, we get
\[ \frac{{\bf a} \cdot {\bf b}}{|{\bf a}|} = |{\bf b}|\,\cos\theta \qquad \iff \qquad \left( \frac{{\bf a}}{|{\bf a}|} \right) \cdot {\bf b} = \hat{\bf e}_a \cdot {\bf b} = |{\bf b}|\,\cos\theta \]
Noting that |b| cos θ is simply the projection of b along a, we conclude that in order to find the perpendicular projection of a vector b along another vector a, take dot product of b with \( \hat{\bf e}_a , \) the unit vector along a.

The dot product of any two vectors of the same dimension can be done with the dot operation given as Dot[vector 1, vector 2] or with use of a period “. “ .

{1,2,3}.{2,4,6}
28
Dot[{1,2,3},{3,2,1} ]
10
Willard Gibbs
With Euclidean norm ‖·‖2, the dot product formula
\[ {\bf x} \cdot {\bf y} = \| {\bf x} \|_2 \, \| {\bf y} \|_2 \, \cos \theta , \]
defines θ, the angle between two vectors. The dot product was first introduced by the American physicist and mathematician Josiah Willard Gibbs (1839--1903) in the 1880s. ■

An inner product of two vectors of the same size, usually denoted by \( \left\langle {\bf x} , {\bf y} \right\rangle ,\) is a generalization of the dot product if it satisfies the following properties:

  • \( \left\langle {\bf v}+{\bf u} , {\bf w} \right\rangle = \left\langle {\bf v} , {\bf w} \right\rangle + \left\langle {\bf u} , {\bf w} \right\rangle . \)
  • \( \left\langle {\bf v} , \alpha {\bf u} \right\rangle = \alpha \left\langle {\bf v} , {\bf u} \right\rangle \) for any scalar α.
  • \( \left\langle {\bf v} , {\bf u} \right\rangle = \overline{\left\langle {\bf u} , {\bf v} \right\rangle} , \) where overline means complex conjugate.
  • \( \left\langle {\bf v} , {\bf v} \right\rangle \ge 0 , \) and equal if and only if \( {\bf v} = {\bf 0} . \)

The fourth condition in the list above is known as the positive-definite condition. A vector space together with the inner product is called an inner product space. Every inner product space is a metric space. The metric or norm is given by

\[ \| {\bf u} \| = \sqrt{\left\langle {\bf u} , {\bf u} \right\rangle} . \]
The nonzero vectors u and v of the same size are orthogonal (or perpendicular) when their inner product is zero: \( \left\langle {\bf u} , {\bf v} \right\rangle = 0 . \) We abbreviate it as \( {\bf u} \perp {\bf v} . \)

If A is an n × n positive definite matrix and u and v are n-vectors, then we can define the weighted Euclidean inner product

\[ \left\langle {\bf u} , {\bf v} \right\rangle = {\bf A} {\bf u} \cdot {\bf v} = {\bf u} \cdot {\bf A}^{\ast} {\bf v} \qquad\mbox{and} \qquad {\bf u} \cdot {\bf A} {\bf v} = {\bf A}^{\ast} {\bf u} \cdot {\bf v} . \]
In particular, if w1, w2, ... , wn are positive real numbers, which are called weights, and if u = ( u1, u2, ... , un) and v = ( v1, v2, ... , vn) are vectors in ℝn, then the formula
\[ \left\langle {\bf u} , {\bf v} \right\rangle = w_1 u_1 v_1 + w_2 u_2 v_2 + \cdots + w_n u_n v_n \]
defines an inner product on \( \mathbb{R}^n , \) that is called the weighted Euclidean inner product with weights w1, w2, ... , wn.
Example 4: The Euclidean inner product and the weighted Euclidean inner product (when \( \left\langle {\bf u} , {\bf v} \right\rangle = \sum_{k=1}^n a_k u_k v_k , \) for some positive numbers \( a_k , \ (k=1,2,\ldots , n \) ) are special cases of a general class of inner products on \( \mathbb{R}^n \) called matrix inner product. Let A be an invertible n-by-n matrix. Then the formula
\[ \left\langle {\bf u} , {\bf v} \right\rangle = {\bf A} {\bf u} \cdot {\bf A} {\bf v} = {\bf v}^{\mathrm T} {\bf A}^{\mathrm T} {\bf A} {\bf u} \]
defines an inner product generated by A.

Example 5: In the set of integrable functions on an interval [a,b], we can define the inner product of two functions f and g as
\[ \left\langle f , g \right\rangle = \int_a^b \overline{f} (x)\, g(x) \, {\text d}x \qquad\mbox{or} \qquad \left\langle f , g \right\rangle = \int_a^b f(x)\,\overline{g} (x) \, {\text d}x . \]
Then the norm \( \| f \| \) (also called the 2-norm or 𝔏² norm) becomes the square root of
\[ \| f \|^2 = \left\langle f , f \right\rangle = \int_a^b \left\vert f(x) \right\vert^2 \, {\text d}x . \]
In particular, the 2-norm of the function \( f(x) = 5x^2 +2x -1 \) on the interval [0,1] is
\[ \| 2 x^2 +2x -1 \| = \sqrt{\int_0^1 \left( 5x^2 +2x -1 \right)^2 {\text d}x } = \sqrt{7} . \]
Example 6: Consider a set of polynomials of degree n. If
\[ {\bf p} = p(x) = p_0 + p_1 x + p_2 x^2 + \cdots + p_n x^n \quad\mbox{and} \quad {\bf q} = q(x) = q_0 + q_1 x + q_2 x^2 + \cdots + q_n x^n \]
are two polynomials, and if \( x_0 , x_1 , \ldots , x_n \) are distinct real numbers (called sample points), then the formula
\[ \left\langle {\bf p} , {\bf q} \right\rangle = p(x_0 ) q(x_0 ) + p_1 (x_1 )q(x_1 ) + \cdots + p(x_n ) q(x_n ) \]
defines an inner product, which is called the evaluation inner product at \( x_0 , x_1 , \ldots , x_n . \) ■
Example 7: What is the angle between i and i + j + 2k?
\begin{align*} \theta &= \arccos \left( \frac{{\bf i} \cdot ({\bf i} + {\bf j} + 2 {\bf k})}{\| {\bf i} \| \cdot \| {\bf i} + {\bf j} + 2 {\bf k} \| } \right) \\ &= \arccos \left( \frac{1}{\sqrt{6}} \right) \approx 1.15026. \end{align*}
========================== to be checked ===============

The dot product of two vectors of the same size \( {\bf x} = \left[ x_1 , x_2 , \ldots , x_n \right] \) and \( {\bf y} = \left[ y_1 , y_2 , \ldots , y_n \right] \) (independently whether they are columns or rows) is the number, denoted either by \( {\bf x} \cdot {\bf y} \) or \( \left\langle {\bf x} , {\bf y} \right\rangle ,\)
\[ \left\langle {\bf x} , {\bf y} \right\rangle = {\bf x} \cdot {\bf y} = x_1 y_1 + x_2 y_2 + \cdots + x_n y_n , \]
when entries are real, or
\[ \left\langle {\bf x} , {\bf y} \right\rangle = {\bf x} \cdot {\bf y} = \overline{x_1} y_1 + \overline{x_2} y_2 + \cdots + \overline{x_n} y_n , \]
when entries are complex.

An outer product is the tensor product of two coordinate vectors \( {\bf u} = \left[ u_1 , u_2 , \ldots , u_m \right] \) and \( {\bf v} = \left[ v_1 , v_2 , \ldots , v_n \right] , \) denoted \( {\bf u} \otimes {\bf v} , \) is an m-by-n matrix W such that its coordinates satisfy \( w_{i,j} = u_i v_j . \) The outer product \( {\bf u} \otimes {\bf v} , \) is equivalent to a matrix multiplication \( {\bf u} \, {\bf v}^{\ast} , \) (or \( {\bf u} \, {\bf v}^{\mathrm T} , \) if vectors are real) provided that u is represented as a column \( m \times 1 \) vector, and v as a column \( n \times 1 \) vector. Here \( {\bf v}^{\ast} = \overline{{\bf v}^{\mathrm T}} . \)

For three dimensional vectors \( {\bf a} = a_1 \,{\bf i} + a_2 \,{\bf j} + a_3 \,{\bf k} = \left[ a_1 , a_2 , a_3 \right] \) and \( {\bf b} = b_1 \,{\bf i} + b_2 \,{\bf j} + b_3 \,{\bf k} = \left[ b_1 , b_2 , b_3 \right] \) , it is possible to define special multiplication, called cross-product:
\[ {\bf a} \times {\bf b} = \det \left[ \begin{array}{ccc} {\bf i} & {\bf j} & {\bf k} \\ a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \end{array} \right] = {\bf i} \left( a_2 b_3 - b_2 a_3 \right) - {\bf j} \left( a_1 b_3 - b_1 a_3 \right) + {\bf k} \left( a_1 b_2 - a_2 b_1 \right) . \]
Example: For instance, if m = 4 and n = 3, then
\[ {\bf u} \otimes {\bf v} = {\bf u} \, {\bf v}^{\mathrm T} = \begin{bmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \end{bmatrix} \begin{bmatrix} v_1 & v_2 & v_3 \end{bmatrix} = \begin{bmatrix} u_1 v_1 & u_1 v_2 & u_1 v_3 \\ u_2 v_1 & u_2 v_2 & u_2 v_3 \\ u_3 v_1 & u_3 v_2 & u_3 v_3 \\ u_4 v_1 & u_4 v_2 & u_4 v_3 \end{bmatrix} . \]
In Mathematica, the outer product has a special command:
Outer[Times, {1, 2, 3, 4}, {a, b, c}]
Out[1]= {{a, b, c}, {2 a, 2 b, 2 c}, {3 a, 3 b, 3 c}, {4 a, 4 b, 4 c}}

Applications in Physics

Vector and scalar products are intimately associated with a variety of physical concepts. For example, the work done by a force applied at a point is defined as the product of the displacement and the component of the force in the direction of displacement (i.e., the projection of the force onto the direction of the displacement). Thus the component of the force perpendicular to the displacement "does no work." If F is the force and s the displacement, then the work W is by definition equal to
\[ W = F_{\parallel} s = F\,s\,\cos\left( {\bf F}, {\bf s} \right) = {\bf F} \bullet {\bf s} . \]
Suppose the force makes an obtuse angle with the displacement, so that the force is "resistive." Then the work is regarded as negative, in keeping with formula above.

 

  1. What is the angle between the vectors i + j and i + 3j?
  2. What is the area of the quadrilateral with vertices at (1, 1), (4, 2), (3, 7) and (2, 3)?
  1. Vector addition
  2. Deay, T. and Manogue, C.A., he Geometry of the Dot and Cross Products, Journal of Online Mathematics and Its Applications 6.