Linear Regression in One Line of Algebra

An interviewer asks: "Fit linear regression without gradient descent." The weights with the smallest squared error solve one equation, the normal equation: (X.T @ X) w = X.T @ y, where .T swaps rows and columns and @ multiplies matrices. Solve it once and you get the exact weights: no learning rate, no loop.

The bias fits in with a trick: put a column of ones in front of X. The weight of that column is added to every prediction, so it is the bias, the first number of w. np.ones((rows, columns)) gives back an array of that shape full of 1.0, and np.hstack([a, b]) gives back the columns of a followed by the columns of b.

np.linalg.solve(A, b) gives back the array w for which A @ w equals b. On another system:

A = np.array([[2.0, 0.0], [0.0, 4.0]])
b = np.array([6.0, 8.0])
print(np.linalg.solve(A, b))   # [3. 2.]: 2 * 3 = 6 and 4 * 2 = 8

On the 6 rows of 2 columns in the starting code, the program must print:

Bias: 3.069
Weights: [1.97  1.509]
Predictions: [11.08 11.54 16.52 13.97 21.97 20.92]
Real values: [11.  11.  17.  14.5 22.  20.5]

Your task: finish fit_normal(X, y): build the column of ones, put it in front of X, and solve the normal equation for w, bias first (the function already returns w). Then write predict(X, w), which returns the bias w[0] plus X times the other weights w[1:]. The hidden check compares your numbers with scikit-learn's LinearRegression.

Assignment
Follow the Coding Tutorial to fit the weights with the normal equation and write predict.

Hint
Use Xb, the data with the ones in front, on both sides of the equation. If one column is a multiple of another, many weights give the same predictions and solve stops with LinAlgError: Singular matrix; the check tries that case too.