Handcrafted algorithms It is a very good exercise to write your algorithms from scratch to see what is really happening. In this post I used matrix notation in R to run linear regression. Simulated data First, we need some data. I wrote a simple function to simulate our X from Y. makeX <- function(Y, n){ X <- matrix ( 1 , ncol= 1 , nrow = dim (Y)[ 1 ]) for (i in 1 :n){ X1 <- Y + rnorm ( 100 , sd = 0.25 ) + sample ( seq (- 1 , 1 , 001 ), 100 , replace = T) X <- cbind (X, X1) } return (X) } One dependent variable We start with a very simple model. Y <- matrix ( rnorm ( 100 ), ncol= 1 ) X <- makeX (Y, 1 ) The cool thing is that we can find the parameters P (i.e. intercept and slope) with one single matrix equation: P = (X t X) -1 X t Y In R this looks like this: P <- solve (( t (X) %*% X)) %*% t (X) %*% Y And now we can run it and plot the results: E <- Y - X %*% P Yhat &l