Naive Bayes from Scratch

An interviewer asks: "Implement Naive Bayes for yes/no features, without a library." The idea in one sentence: for each class, multiply its share of the data by the chance of each of the new row's feature values in that class, and pick the class with the bigger result.

Training is counting. For each class you keep its prior, the share of rows in that class, and for each feature its rate, the share of the class's rows with a 1 there. With smoothing you add 1 to the count of 1s and 2 to the number of rows, so no rate is ever 0: a single 0 would wipe out the whole product. A class of 3 rows that never has a 1 in a feature gets the rate (0 + 1) / (3 + 2) = 0.2 there, not 0.

Predicting: a feature that is 1 brings its rate, a feature that is 0 brings 1 − rate. Multiplying many chances gives tiny numbers, so you add their logs instead: np.log gives back the natural log of every item of an array, and the bigger sum of logs belongs to the bigger product. .sum(axis=0) adds up each column, .sum(axis=1) each row, and np.argmax gives back the position of the biggest item:

a = np.array([[1, 0], [1, 1], [0, 1]])
print(a.sum(axis=0))                      # [2 2]
print(a.sum(axis=1))                      # [1 2 1]
print(np.argmax(np.array([-3.2, -2.9])))  # 1

The starting code has 10 customers in X with 3 yes/no features each (the middle one is returned_item), their labels y, 0 or 1, a new_customer, and the frames of fit_nb(X, y) and predict_nb(x, priors, rates). Your code must print:

Class shares: [0.4 0.6]
Rates of a 1, per class:
[[0.333 0.667 0.333]
 [0.75  0.125 0.75 ]]
New customer [1 1 1] -> 1

Your task: in fit_nb, fill in each class's share and its smoothed rates; in predict_nb, take the logs of the chances for the features that are 1 and for those that are 0, and score each class: the log of its share plus the sum of those logs. Use numpy only: the check compares with scikit-learn's BernoulliNB.

Assignment
Follow the Coding Tutorial to fill in fit_nb and predict_nb one line at a time and classify the new customer.

Hint
rows holds the customers of class c, so its share is len(rows) / len(X). Where x is 1, 1 - x is 0, so multiplying by x or by 1 - x keeps only the features that are 1 or only those that are 0.