Start learning free

LESSON 74 OF 165 · MACHINE LEARNING: ML INTERVIEW QUESTIONS

Accuracy, Precision, Recall from Scratch

Practice

A classic interview question: write a function that gives the accuracy, precision and recall of a model's guesses, without scikit-learn. Each guess is 1 (the customer will buy again) or 0. A true positive is a 1 the model said and got right, a false positive is a 1 it said wrongly, and a false negative is a real 1 it missed.

Accuracy is the share of all guesses that are right. Precision is tp / (tp + fp): of the 1s the model said, the share that were right. Recall is tp / (tp + fn): of the real 1s, the share the model found.

You count with masks. Comparing a numpy array with a number, like ages > 30, gives back an array of True and False, one per item. & joins two masks into one that is True only where both are True (keep each comparison in its own parentheses). .sum() gives back how many Trues there are, and .mean() gives back their share:

ages = np.array([25, 34, 41, 38])
print(((ages > 30) & (ages < 40)).sum())   # 2
print((ages > 30).mean())                    # 0.75

One special case: a model that never says 1 has tp + fp = 0, and 0 / 0 is no score at all. Divide by max(tp + fp, 1) instead: Python's max gives back the larger number, and tp is 0 then too, so precision is 0. Recall gets the same guard. With the lesson's 10 customers, your code and the check print:

Accuracy: 0.7
Precision: 0.8
Recall: 0.667
Never says 1: accuracy 0.4 precision 0.0 recall 0.0
No real 1s: accuracy 0.75 precision 0.0 recall 0.0
Same as scikit-learn on 30 random label sets: 30 of 30

Your task: inside scores, above the return, count tp, fp and fn with masks, then set accuracy, precision and recall.


Assignment
Follow the Coding Tutorial to count the true positives, false positives and false negatives, then turn them into accuracy, precision and recall.


Hint
Each count is one line: two comparisons in parentheses joined with &, then .sum(). Accuracy compares the two arrays directly. The max(…, 1) keeps the bottom of each fraction at least 1.


STEP 1 OF 6
    Ctrl + Enter to check