Project: Pick the Best Model

In this module you trained several kinds of models. Now you put four of them side by side on one question: will a customer buy again? A fair race uses the same split: every model learns from the same 225 training customers and is scored on the same 75 test customers. The first model is a baseline: DummyClassifier(strategy="most_frequent") ignores the features and always gives back the most common answer of the training rows, here 1 (yes). A real model has to beat it.

Three scores: each of these functions compares the real answers with the predicted ones and gives back one number between 0 and 1. accuracy_score is the share of answers that are right. precision_score: of the customers the model said yes to, the share that really bought again. recall_score: of the customers who really bought again, the share the model said yes to.

from sklearn.metrics import accuracy_score, precision_score, recall_score

real      = [1, 1, 1, 1, 0]
predicted = [1, 1, 0, 1, 1]
print(accuracy_score(real, predicted))   # 0.6: 3 of the 5 answers are right
print(precision_score(real, predicted))  # 0.75: 3 of the 4 yes answers are right
print(recall_score(real, predicted))     # 0.75: 3 of the 4 buyers got a yes

The starting code reads customers.csv, splits it and keeps the four models in the dict models, name to model. You loop over the names, fit and score each model, and keep one list per model in rows. pd.DataFrame(rows, columns=[...]) gives back a table with one row per inner list, and .round(3) rounds its numbers to 3 digits. table.sort_values("accuracy", ascending=False) gives back the table with the biggest accuracy first, and .iloc[0] gives back its first row.

Your script must print this table and one line with your pick:

              model  accuracy  precision  recall
           baseline     0.653      0.653   1.000
logistic regression     0.733      0.809   0.776
               tree     0.720      0.833   0.714
             forest     0.760      0.804   0.837
Pick: forest - best accuracy 0.76 vs the baseline's 0.653

Your task: below rows = [], loop over models: fit each model on the training rows, predict the test rows and append its name and three scores to rows. Then make the rounded table, print it without the row numbers, keep the row with the best accuracy in best and print your pick next to the baseline's accuracy.

Assignment
Follow the Coding Tutorial to score four models on the same test customers, print one table of their scores and print the model you pick.

Hint
Looping over a dict gives back its keys, the four names, and models[name] is the model for that name. The three lines inside the loop are indented 4 spaces; the table and the two prints come after the loop, not indented. The baseline is row 0 of table, so its accuracy is table["accuracy"][0].