The first condition says that transforming a sum is the same as transforming each vector first and then adding. The second condition says that scaling before transforming is the same as transforming first and then scaling.

In finite-dimensional spaces, linear transformations are represented by matrices. For example, let

Then the transformation T(x) = Ax is

This transformation keeps the $y$-coordinate fixed but changes the $x$-coordinate, so it acts like a horizontal shear combined with stretching.

For a concrete example,

Geometrically, a matrix tells us where the standard basis vectors go:

For the matrix A above,

Once we know where e_1 and e_2 go, we know how every vector in the plane is transformed.

For example:

Other useful norms are the L1 and L infinite:

for example:

The distance between two vectors is the norm of their difference:

using the Euclidian norm we have:

Geometrically the norm is the length of the vector from the origin and while the distance is how far apart the vectors are from each other.

As we discussed on the previous post, when creating a matrix we can use the combination of row vectors or column. however how does a matrix look like in the euclidean 2d space. Here we can see a diagram of what a matrix looks like:

Pairwise multiplication, also called the Hadamard product, multiplies corresponding entries:

We can also get the dot product of a vector, such as:

A tensor is a mathematical object that generalizes a scalar, vector and matrices in higher dimension. Think of matrices in higher dimensions.

This concludes the article. As we progress further into the course, we will define and explore more linear algebra concepts needed for AI research.

In this guide, I will cover some of the mathematics and code behind algebraic operations, which form the backbone of modern Machine Learning (ML).

The table of contents for this guide is as follows:

• Scalars, Vectors, Matrices, and Tensors

• Matrix Operations

• Dot Product

• Norms and Distances

• Linear Transformations

• Eigenvalues and Eigenvectors

• Matrix Factorization

• Singular Value Decomposition, SVD

• Orthogonality

• Projections

• Rank

• Positive Definite Matrices

Scalars, Vectors, and Matrices

In linear algebra, we encounter different algebraic notations. For example, vectors consist of both magnitude and direction. They can be added together and multiplied, or scaled, by a number called a scalar.

Here are some examples of a vector.

Here is what happens when vector u is scaled by scalar c:

Here is what happens when vector u is added to vector v:

Vectors can be represented as column vectors, such as u and v, or as row vectors, like this:

Vector d is a vector in two-dimensional Euclidean space, and it looks something like this:

Vector d can be compressed using the scalar 0.5, and visually, it looks something like this:

The addition of vector d and (2, 3) results in the vector (5, 5), which looks something like this:

We can think of a matrix as a collection of row vectors:

It can also be viewed as a collection of column vectors:

This concludes our tutorial on vectors, scalars, and matrices. More details about matrices will be discussed in future articles. Stay tuned.

In this series, I will provide a deep dive into mathematics, research basics, and, finally, insights into state-of-the-art and classic research papers. Here is a list of things you need to know in order to become a fully fledged AI researcher, some of which you may already know.

One caveat: AI is a huge domain, and you must be specific about the topic you want to explore in order to become world-class in it. I am certainly not world-class in anything yet, but I am trying my best to get there. :-)

My Personal Roadmap

Mathematics

• Linear algebra

• Probability and statistics

• Information theory

• Optimization

• Calculus

Coding

Python

• Syntax

• Basic keywords

• Basic built-in functions

• Conditionals

• Object-oriented programming

ML Frameworks

• PyTorch

• TensorFlow

• scikit-learn

Other Libraries

• NumPy

• Matplotlib

• SciPy

• Seaborn

Machine Learning Fundamental's

• Supervised learning

• Unsupervised learning

• Semi-supervised learning

• Reinforcement learning

• Loss functions

• Regularization

• Overfitting and underfitting

• Cross-validation

• Bias–variance tradeoff

• Feature engineering

• Evaluation metrics

Deep Learning

• Neural networks

• Backpropagation

• Activation functions

• Initialization

• Normalization

• CNNs

• RNNs / LSTMs

• Transformers

• Attention mechanisms

• Autoencoders

• Diffusion models

• Generative models

Research Practice

• Read papers consistently

• Reproduce important papers

• Build small projects

• Run controlled experiments

• Write technical notes

• Share code publicly

• Learn to communicate results clearly

• Develop taste for good research questions

Tools

• Git and GitHub

• Linux / command line

• Jupyter notebooks

• Docker

• Conda / virtual environments

• Weights & Biases or TensorBoard

• Slurm or cluster computing basics

• Cloud platforms such as AWS, GCP, or Azure

Fluency in English

• Reading

• Writing

Domains that you can specialize in after learning the basics are as follows:

• Computer vision

• Natural language processing

• Reinforcement learning

• Robotics

• Generative AI

• Multimodal learning

• AI safety

• Interpretability

• Causal inference

• Graph machine learning

• Recommendation systems

• AI for science

• Human-AI interaction

• Bias and fairness

• Privacy

• Security

• Model misuse

• Interpretability

• Alignment

• Evaluation of harmful outputs

• Responsible deployment

Next, we will explore the topic of linear algebra, as it is the backbone of machine learning. I will cover the necessary concepts and skills needed to understand vectors, matrices, and their operations, which form the foundation of modern AI systems.