Pipelines

In the last lesson you fitted the scaler on the training rows only. Cross-validation makes that harder: cross_val_score cuts the rows into 5 folds and lets each fold be the test rows once. If you scale all 300 rows first, the scaler has already seen every round's test rows. The starting code does exactly that and prints a mean of 0.723.

A pipeline joins the steps into one object. make_pipeline(step1, step2) gives back a model that runs the steps in order: fit fits the scaler on the rows it gets, scales them and fits the model on the result; score and predict only scale with what the scaler already learned, then let the model answer. pipeline.named_steps holds the steps by name, and make_pipeline names each one after its class, in lowercase.

model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
model.fit(X_train, y_train)
print(model.predict(X_test[:3]))

Because a pipeline is one model, you can hand it to cross_val_score with the raw X. Every round fits the whole pipeline again on that round's training folds, so the scaler never sees the round's test fold. Your script must print:

Scaled first, cross-validated: [0.65  0.717 0.733 0.717 0.8  ] mean 0.723
Steps: ['standardscaler', 'kneighborsclassifier']
Pipeline, test rows: 0.64
Pipeline, cross-validated: [0.65  0.717 0.733 0.717 0.783] mean 0.72

Only the last round differs, 0.8 against 0.783, and only the pipeline's mean, 0.72, comes from a fair test.

Your task: join a StandardScaler() and a KNeighborsClassifier(n_neighbors=5) with make_pipeline in pipeline and print its step names; fit it on the raw training rows and print its score on the test rows; then cross-validate it on X and y with cv=5 and print the five scores and their mean.

Assignment
Follow the Coding Tutorial to build the pipeline, score it on the test rows and cross-validate it.

Hint
Give the pipeline the raw rows (X_train, X_test, X), never X_scaled: scaling is now the pipeline's own first step.