ML Math - Introduction to Linear Algebra

Linear Algebra is the branch of mathematics that focuses on the study of vectors, matrices, and the linear relationships between them. While a vector is a one-dimensional array of numbers, a Matrix is a two-dimensional (2-D) grid of numbers arranged in rows and columns. We define a matrix by its dimensions, specifically the number of rows ( $m$ ) and the number of columns ( $n$ ).

Mathematically, a matrix $A$ is represented as follows:

\mathbf{A} \in \mathbb{R}^{m \times n}

Visually, the elements or individual item of a matrix are indexed by their row position ( $i$ ) and column position ( $j$ ):

\mathbf{A} = \begin{bmatrix} a_{1,1} & a_{1,2} & \cdots & a_{1,n} \\ a_{2,1} & a_{2,2} & \cdots & a_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m,1} & a_{m,2} & \cdots & a_{m,n} \end{bmatrix}

The matrix $A$ above has a dimension of ( $m \times n$ ) and $a$ represents the individual elements of the matrix, now to locate each individual element, we can do it as $a_{i,j}$ where $i$ denotes the row index and $j$ denotes the column index. This system works like a coordinate grid, allowing us to pinpoint the exact location of any value within the matrix.

For example, if we want to find the element located specifically in the second row and the third column, we would identify it as:

a_{2,3}

By using this notation, we can describe the position of any number in a matrix of any size. If a matrix has rows $m$ and $n$ columns, then:

1 \leq i \leq m

1 \leq j \leq n

Special Types of Matrices

In Linear Algebra, we classify matrices based on their dimensions and the arrangement of their elements. Understanding these specific types is essential for performing advanced operations.

1. Square Matrix

A Square Matrix is a matrix that has the same number of rows and columns ( $m = n$ ). :

\mathbf{A} = \begin{bmatrix} a_{1,1} & a_{1,2} \\ a_{2,1} & a_{2,2} \end{bmatrix}

In this example, the matrix has a dimension of ( $2 \times 2$ )

2. Row and Column Matrices (Vectors)

We can represent the vectors as matrices with a single row or a single column.

A Column Matrix has a dimension of ( $m \times 1$ ).

\text{Column Matrix: } \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \in \mathbb{R}^{3 \times 1}

A Row Matrix has a dimension of ( $1 \times n$ ).

\text{Row Matrix: } [ x_1 \quad x_2 \quad x_3 ] \in \mathbb{R}^{1 \times 3}

Introducing Matrix Operations

Because matrices are structured grids, we cannot treat them like simple numbers. There are specific rules for how they interact with each other. In the following sections of this series, we will dive deep into these operations, but here is a brief overview of what they allow us to do:

Addition and Subtraction: Allows us to combine or compare two datasets of the same shape.
Scalar Multiplication: Allows us to scale every data point in a matrix by a single factor.
Matrix Multiplication: The process of calculating the relationship between two matrices (this is how neural network layers process information).
Transposition: Flipping a matrix over its diagonal to change how rows and columns are oriented.

Summary

Linear Algebra is the mathematical study of vectors and matrices, providing the rules for how these objects interact with each other.
A Matrix is a two-dimensional (2-D) grid of numbers (scalars) organized into $m$ rows and $n$ columns, representing a collection of data.
Every value inside a matrix is called an element, and we locate it using the notation $a_{i, j}$ , where $i$ is the row and $i$ is the column.
Linear Algebra allows us to scale our math by moving from single data points (vectors) to entire datasets (matrices) that can be processed at once.
To work with matrices, we use specific Matrix Operations, such as addition, scalar multiplication, and transposition, to transform and manipulate the data.