Start learning free

LESSON 87 OF 165 · MACHINE LEARNING: ML INTERVIEW QUESTIONS

AUC Without a Library

Practice

An interviewer asks: "Your model gives every customer a score. Compute the ROC AUC with numpy only, no scikit-learn." The idea in one sentence: the AUC is the share of (real 1, real 0) pairs in which the real 1 gets the higher score, and a tie counts as half a win. 1.0 means every 1 is ranked above every 0, 0.5 is no better than a coin flip, and 0.0 means the order is reversed.

A small example: the real 1s score 0.8 and 0.5, and the real 0s score 0.6 and 0.2. That makes 2 × 2 = 4 pairs. 0.8 beats both 0s and 0.5 beats only 0.2, so 3 of the 4 pairs are won: AUC = 3 / 4 = 0.75. numpy compares one number with a whole array at once and gives back one True or False per item; .sum() gives back how many are True:

zeros = np.array([0.6, 0.2])
print(0.5 > zeros)          # [False  True]
print((0.5 > zeros).sum())  # 1

The starting code has the labels y_true, the model's scores and a function auc that only sets wins = 0.0. A mask picks items: scores[y_true == 1] gives back the scores of the rows whose label is 1 (the code already prints it), and len(a) gives back how many items a has. When auc is done, your code and the check print:

Scores of the 1s: [0.9 0.6 0.4 0.8]
Scores of the 0s: [0.3 0.4 0.1 0.7]
AUC: 0.844
scikit-learn's roc_auc_score: 0.844
Perfect order: 1.0
Reversed order: 0.0
Every score the same: 0.5
Scores times 100, same order: 0.844
Same as roc_auc_score on 30 random score sets with ties: 30 of 30

Your task: fill in auc. Put the scores of the real 1s in pos and those of the real 0s in neg, loop over the positive scores, add to wins how many negatives score lower and half of how many score the same, and return wins divided by the number of pairs.


Assignment
Follow the Coding Tutorial to write auc: the share of (real 1, real 0) pairs the model ranks the right way.


Hint
There are len(pos) * len(neg) pairs: 4 × 4 = 16 here. The real 1 scored 0.4 ties with the real 0 scored 0.4, so that pair adds 0.5 to wins.


STEP 1 OF 6
    Ctrl + Enter to check