k-Nearest Neighbors from Scratch

A classic interview question: "Implement k-nearest neighbors with numpy only. You get training rows, their labels and one new row: predict the new row's label." The idea in one sentence: find the k training rows closest to the new row and return the label most of them have.

Distances to every row at once: numpy broadcasting subtracts one row from every row of a 2D array, so X - point gives back an array with the same shape as X. ** 2 squares every number, .sum(axis=1) adds up each row (one number per row) and np.sqrt takes the square root. Together they give the straight-line distance from each row to the point.

Picking the neighbors and counting votes: np.argsort(a) gives back the positions of the values, from the smallest value to the largest, and [:k] keeps the first k. Indexing with an array of positions, like y[positions], gives back the values at those positions. np.unique(a, return_counts=True) gives back two arrays: the different values, sorted, and how many times each one appears. np.argmax(a) gives back the position of the biggest value (the first one if there is a tie). Here they are on other values:

import numpy as np

grid = np.array([[0, 0], [3, 4], [1, 1]])
print(grid - np.array([1, 1]))   # each row minus [1, 1]: [-1 -1], [2 3], [0 0]

scores = np.array([4.0, 1.0, 3.0])
print(np.argsort(scores))        # [1 2 0]: position 1 holds the smallest
print(np.argsort(scores)[:2])    # [1 2]: the 2 smallest

words = np.array(["b", "a", "b"])
values, counts = np.unique(words, return_counts=True)
print(values, counts)            # ['a' 'b'] [1 2]
print(np.argmax(counts))         # 1: the biggest count sits at position 1

The starting code has 9 training customers in X_train (two numbers each), their labels in y_train ("no" or "yes") and a new_customer at [5, 3]. It already prints the prediction for k = 1 and k = 3, which must be:

k=1: yes
k=3: no

The check then calls your function with k = 9 (every customer votes, so the answer is yes), on rows of three numbers, and on 40 random points, where it must agree with scikit-learn's KNeighborsClassifier.

Your task: below new_customer, write knn_predict(X, y, point, k): the differences, the distances, the positions of the k closest rows, their labels, the label counts, and finally return the most common label.

Assignment
Follow the Coding Tutorial to write knn_predict one line at a time, from the subtraction to the vote.

Hint
You need the positions from np.argsort, not the sorted distances: positions are what you use to look up labels in y. The function only returns at the last step, so the script prints None until then.