Gradient Boosting

A random forest grows many deep trees side by side: each one learns on its own, and then they vote. Gradient boosting grows small trees one after another. The first tree makes a rough guess, each new tree learns what the trees before it still get wrong, and the model adds up all their answers.

The model is GradientBoostingClassifier from sklearn.ensemble, already imported in the starting code. n_estimators is how many trees it builds, and max_depth is how many questions each tree may ask in a row. fit builds the trees, one after another; predict gives back one guess per row, and score(X, y) gives back the share of rows it gets right. A tiny example on made-up study hours:

hours = [[1], [2], [3], [4], [5], [6]]
passed = [0, 0, 0, 1, 1, 1]
model = GradientBoostingClassifier(n_estimators=10, max_depth=1, random_state=0)
model.fit(hours, passed)
print(model.predict([[2], [5]]))   # [0 1]

The starting code loads customers.csv, splits it into 225 training and 75 test rows, and scores a random forest of 50 trees. You add a boosting model next to it. Below your code, the check prints how many trees your model has and what share of the training rows it gets right with 1, 10 and all 50 of them. Your script must print:

Random forest, test rows: 0.76
Gradient boosting, test rows: 0.747
Trees in the model: 50 | depth of each: 2
Trees used: 1 | right on the training rows: 0.578
Trees used: 10 | right on the training rows: 0.844
Trees used: 50 | right on the training rows: 0.871
Random forest, right on the training rows: 1.0

Out of 75 test customers, 0.76 is 57 right and 0.747 is 56: one customer apart. The forest learns its training rows by heart (1.0); boosting doesn't, and does about as well on new customers.

Your task: make a GradientBoostingClassifier with n_estimators=50, max_depth=2 and random_state=0 and keep it in boost, fit it on the training rows, and print "Gradient boosting, test rows:" with its score on the test rows, rounded to 3 digits.

Assignment
Follow the Coding Tutorial to build, fit and score a gradient boosting model next to the random forest.

Hint
Copy the shape of the forest's three lines: make the model, fit it on X_train and y_train, then print its score on X_test and y_test, rounded to 3 digits.