Stop When It Stops Helping

In the last lesson you read the test scores to see how many trees were enough. With early stopping the model finds that out by itself while it trains, without touching the test rows. It holds back some of its training rows, adds trees one at a time, checks each new tree on the held-back rows, and stops when new trees no longer do better there.

Two settings of GradientBoostingClassifier turn it on. validation_fraction=0.2 holds back a fifth of the training rows, and n_iter_no_change=5 stops when 5 new trees in a row don't do better on them. n_estimators becomes the most trees it may build. After fit, n_estimators_, with the underscore at the end, gives back how many trees it really built. For example, with a much higher ceiling:

roomy = GradientBoostingClassifier(n_estimators=1000, n_iter_no_change=10, validation_fraction=0.1, random_state=0)
roomy.fit(X_train, y_train)
print(roomy.n_estimators_)   # how many of the 1000 it built before stopping

The starting code trains a model with all 200 trees of depth 2 and prints its test score. You add the same model with early stopping, then print how many trees it kept and its test score. Below your code, the check prints how the training rows were shared out. Your script must print:

All 200 trees, test rows: 0.747
Trees kept: 52
Early stopping, test rows: 0.773
Training rows its trees learned from: 180 of 225
Rows held back to judge each new tree: 45

The early model keeps 52 trees, about a quarter of 200, and gets 58 of the 75 test customers right against 56: as good or a little better, for a quarter of the work.

Your task: make the same model as full plus n_iter_no_change=5 and validation_fraction=0.2 and keep it in early, fit it on the training rows, print "Trees kept:" and early.n_estimators_, then print "Early stopping, test rows:" and its test score, rounded to 3 digits.

Assignment
Follow the Coding Tutorial to train a boosting model that stops by itself and compare it with the one that uses all 200 trees.

Hint
Copy the arguments of full and add the two new ones. n_estimators_ exists only after fit, so print it below the fit line.