I've been reading this document. The goal is to find projection of $b$ on the line $L$ which is determined by vector $a$. ($Proj_L(b)$)
In the document, It is mentioned that If $p$ is thought as approximation to $b$, then $e=b-p$ is the error in that approximation (the word approximation is little confusing though, Isn't $e$ the exact value? Although this is not the problem).
Then we know that If $p$ lies on the line through $a$, then $p = xa$ for some $x$. We also know that, $p$ is orthogonal to $e$, therefore their dot product equates to zero:
$a^T(b-xa)=0$
$a^Tb - a^Txa = 0$
$xa^Ta = a^Tb$
$x = \frac{a^Tb}{a^Ta}$
Solving for $p$:
$p = ax = a\frac{a^Tb}{xa^Ta}$
First part was almost completely understandable, but in the second part this projection is written in the terms of projection matrix ("$P: p = Pb$"):
$p = xa = \frac{aa^Ta}{a^Ta}$
where did $b$ go? $x$ and $a$ have changed places, but isn't dot product commutative?
Then, $P$ is solved:
$P = \frac{aa^T}{a^Ta}$
Somehow, $aa^T$ is 3x3 matrix.
How was this concluded? From my knowledge, The general definition of projection matrix is $A (A^{T}A)^{-1} A^T \vec{x}$ (where $A$ is matrix)
Does the definition above has any relations with the projection matrix that was represented in document? If not, how was it derived?
Thank you!