Eigenvalues and Eigenvectors

As previously discussed, every matrix represents a linear transformation that maps vectors from ℝⁿ to ℝᵐ. If the matrix is square (n = m), vectors remain in the same space after the transformation. In this case, we can study how the transformation stretches, rotates, and skews ℝⁿ.

For any square matrix A, there exist special directions such that vectors lying along these directions are only stretched or shrunk, while their direction remains unchanged. These special directions are called eigenvectors of A.

Eigenvectors represent the intrinsic directions of a linear transformation and reveal its internal structure. If v is an eigenvector of matrix A, then

where λ is a scalar that indicates how much the vector v is stretched or shrunk. This scalar is called the eigenvalue associated with the eigenvector v. This contains two unknowns: v and λ. The trick is to first try to find λ by solving the following equation.

Since, v is not zero, to make sure this equation has a solution, the null space of (A – λI) should be non-trivial. This requires:

This equation is called the characteristic equation of the matrix A. Solving it yields the eigenvalues. once λ is known, we solve (A – λI)v = 0 to find the corresponding eigenvectors.

A Side Note on Determinants

It is useful have a physical interpretation about determinant, to better understand the where the characteristic equation came from.

Determinant is a single number that tells us how much a matrix changes the "volume" of space. For example,

• if det(A) = 5, it means the matrix makes shapes 5 times bigger.

• if det(A) = 1, it means the matrix moves things but keeps the volume the same.

• if det(A) = 0, it means the matrix has squashed space down into a lower dimension (a flat line or a point). Its volume is zero.

Thus, the characteristic equation states that for certain values of λ, the matrix (A – λI) collapses space along some direction. That collapsed direction is precisely the eigenvector.

For an n×n matrix, the characteristic equation yields a polynomial of degree n. So, there exists n eigenvalues. But if some eigenvalues are repeated, you may be in trouble!

For example, if we get (1 – λ)(2 + λ)² = 0 for a 3x3 matrix, λ = -2 is the repeated eigenvalue:

• Algebraically, λ = -2 is repeated two times. So Algebraic Multiplicity, AM = 2.

• Geometrically, the dimensions of the null space of the matrix (A + 2I), called the Geometric Multiplicity (GM), depends on how many linearly independent eigenvectors you have.

As long as AM = GM, it won’t be a problem, but for example, If AM = 2, but GM = 1 (the null space of (A + 2I) is only a line), then a dimension is lost.

Note that the null space is still perfectly well-defined. The "problem" is that for a 3x3 matrix, you want 3 eigenvectors to build a new basis for ℝ³, but now you are short of one direction. Therefore, you cannot decompose your matrix A fully in terms of its eigenvectors.

And now we are missing it.

But there is more than that. This algebraically "wrong prediction" is the mathematics' way of warning that there is a hidden coupling between dimensions that will cause linear growth. For example, consider a set of differential equations, ẋ = Ax. When the matrix A has a full set of eigenvectors, the solution is a clean sum of exponential modes:

When eigenvectors are missing, additional polynomial factors such as t.exp(λt) appear. That extra t means the system does not simply oscillate or decay—it grows linearly in amplitude over time. In physical systems, this often corresponds to resonance-like drift, energy accumulation, or structural or electrical instability; it grows wider and wider until the system fails.

For this reason, engineers love their matrices to be Symmetric to ensure they always have a full set of eigenvectors (AM = GM). Symmetry often arises from physical laws, such as Newton’s third law, which enforces relations like σ₁₂ = σ₂₁ in homogeneous materials. However, even if matrix A is not symmetric, the matrix A^TA is always symmetric. This fact is the foundation of diagonalization methods and SVD.

Yet a symmetric matrix and a full set of eigenvectors still do not guarantee the stability of solution. The stability of a system of equations tightly depends on the eigenvalues of that system.

On the Interpretation of Eigenvalues

First of all, note that to understand the physical meaning of eigenvalues of a system, look at the units. For example, for a set of differential equations ẋ = Ax, if x represents position, then for the eigenvectors of the system we have:

So, λ have the units of 1/Time (Frequency) and it tells us how fast or how often something happens.

Secondly, determine the nature of the system of equations. In engineering, most of the matrices we see generally fall into two categories: either our system of equations is static, or dynamic. this is very crucial when evaluating the eigenvalues, because they are interpreted differently for different type of systems.

Group 1: Static Systems (Av = f)

In static systems, the matrix A represents a fixed relationship between an input and an output.

Example: The "Stiffness" Matrix (Elasticity and Structural Mechanics)

A common static model is: Kx = f

The matrix represents how a structure resists force (F = kx).

• The Vector x: Displacement (how much parts move).

• The vector f: applied force

• The matrix K: Stiffness (how strongly the structure resists deformation).

• The eigenvalue λ: Stiffness magnitude in a principal direction.

Physical Interpretation of Eigenvalues in Statics

1.      λ > 0: Stable Equilibrium (Energy Minimum)

• Moving in that direction increases potential energy.

• The structure resists deformation and pushes back.

• Large λ → stiff direction; small λ → soft direction.

• The Analogy: A marble in a bowl. The more positive the λ, the "steeper" the bowl and the faster the system returns to rest

2.      λ = 0: Rigid-Body Motion or Mechanism

• No restoring force in that direction.

• Unconstrained motion.

• The Analogy: a marble on a plate. If you push the marble, it drifts freely. There is no restoring force.

3.      λ < 0: Static Instability (Buckling)

• Energy decreases when moving away from equilibrium.

• The system wants to deform spontaneously (e.g., column buckling).

• The Analogy: A marble on an upside-down bowl. Any small disturbance causes it to roll away faster and faster.

Other examples: Thermal Conductivity Matrix, The Stress and Strain Tensors, The Mass (Inertia) Matrix

Group 2: Dynamic Systems (ẋ = Ax)

In dynamic systems, the matrix A describes time evolution.

Example: The "State-Space" Matrix (Vibrations/Control)

A simple mass–spring–damper system is:

It can be rewritten as:

• The Vector v: The State (a "stack" of the current position and velocity).

• The eigenvalue λ: Represents the Growth/Decay Rate (real part) and the Frequency (imaginary part) of the mode.

Physical Interpretation of Eigenvalues in Dynamics

1.      Re(λ) < 0: Stable decay (Damping)

• Motion decays exponentially

• Energy is dissipated (e.g., shock absorbers).

• The Analogy: A marble in a bowl. The more negative the λ, the "steeper" the bowl and the faster the system returns to rest.

2.      Re(λ) = 0: Marginal Stability

• Motion neither grows nor decays.

• Move freely without constraint.

• The Analogy: a marble on a plate. If you push the marble, it drifts freely. There is no restoring force.

3.      Re(λ) > 0 (pure imaginary): Instability (Energy Injection)

• Motion grows exponentially.

• Represents uncontrolled vibration.

• The Analogy: A marble on an upside-down bowl. Any small disturbance causes it to roll away faster and faster.

4.      λ = ±ci (Imaginary): Perpetual Orbit

• Pure oscillation with frequency c.

• No damping or growth; perpetual motion in idealized systems.

• The "Imaginary" Logic: Imaginary eigenvalues represent phase shifts between position and velocity. When at the max position, speed is zero, and vice versa.

5.      λ = a ± bi (Complex): The "Spiral"

• Rotation plus growth/decay.

• a < 0: decaying spiral (stable vibration)

• a > 0: growing spiral (dangerous oscillatory instability).

Other examples: Almost all dynamic systems—regardless of the order of their differential equations—can be written in the first-order form of ẋ = Ax by introducing additional state variables.

So, as you see, there is a fundamental contrast between static and dynamic systems in how eigenvalues relate to stability. This difference arises because these systems are governed by different mathematical principles: static systems are associated with energy minimization, whereas dynamic systems are governed by time evolution.

Eigenvalues do not have a universal physical meaning, and their interpretation depends on the form of the governing equations. However, as long as the system is linear and time-invariant (for example, no asymmetric matrices, or no time-varying matrices)—which covers most practical engineering applications—the interpretation rules discussed for static and dynamic systems remain valid and reliable.

Even for nonlinear systems, ẋ = f(x), we introduce the Jacobian matrix to linearize the equations, so we can check stability using eigenvalues. Moreover, partial differential equations (PDEs) can also be discretized into large finite-dimensional linear dynamic systems. Therefore, almost every engineering problem ultimately reduces to a system of algebraic or differential equations whose qualitative behavior is governed by eigenvalues and eigenvectors.