Random Forests

One deep tree learns its training customers by heart, so it makes its own odd mistakes on new ones. A random forest grows many trees and lets them vote: for each customer, the forest gives the answer most of its trees gave. To make the trees different, each one learns from its own random sample of the training rows, drawn with replacement, so some customers come in twice and others not at all. (At each question, a tree also looks at only a random few of the columns.)

RandomForestClassifier(n_estimators=50, random_state=0) makes a forest of 50 trees; random_state=0 makes the random samples the same on every run. You fit and score it like a single tree. forest.predict_proba(rows) gives back one row per customer with two numbers, the chance of 0 and the chance of 1. For a forest, the chance of 1 is the share of its trees that voted yes.

small = RandomForestClassifier(n_estimators=10, random_state=0)
small.fit(X_train, y_train)
print(small.predict_proba(X_test[:2]))

That prints 2 rows of 2 numbers, one row per customer. [:, 1] keeps column 1, the share saying yes, from every row, and .round(2) rounds each number to 2 digits.

This lesson's code compares one tree with a forest on the test rows. The lesson's check adds the last three lines:

One tree: 0.667
Forest of 50 trees: 0.76
Share of trees saying yes: [0.86 0.72 0.26 0.06 0.2 ]
Trees in the forest: 50
Different rows behind tree 1: 145 of 225
One of its trees alone, on average: 0.659

Your task: the split and a single deep tree, tree, are already there. Below them, make a forest of 50 trees with random_state=0 in forest, fit it on the training rows, print its test score rounded to 3 digits, then print the share of trees saying yes for the first 5 test customers.

Assignment
Follow the Coding Tutorial to grow a forest of 50 trees and compare its test score with a single tree's.

Hint
forest.predict_proba(X_test[:5]) has one row per customer and two columns; [:, 1] takes the second column. 0.86 means 43 of the 50 trees said yes.