Similar Products

Counting the customers who bought two products together has a flaw: a product that everyone buys gets big counts next to every other product. In the shop that is the Gift card: 76 customers bought it with the Teapot, more than the 64 who bought the Mug with it. Cosine similarity fixes this by dividing the count by how popular both products are:

similarity = bought both / √(buyers of A × buyers of B)

It is 1.0 when two products have exactly the same buyers and 0.0 when they share none. A toy bakery: Bread and Butter have 4 buyers each and share 3, so 3 / √(4 × 4) = 0.75. A Bag has 16 buyers and shares all 4 of Bread's, but 4 / √(4 × 16) = 0.5. The Bag shares more buyers, yet Butter is closer to Bread.

New functions: cosine_similarity(table), from sklearn.metrics.pairwise, compares every row of a table with every other row and gives back a square numpy array of scores. Your matrix has one row per customer, so give it matrix.T, the turned table with one row per product: you get an 8 by 8 array. pd.DataFrame(array, index=names, columns=names) puts the product names back on it. On one column (a Series), .drop(label) gives back a copy without that row, .idxmax() gives back the label of the largest value, and .max() gives back the value itself.

Your code prints the table rounded to 2 digits, then each product's closest other product:

product    Gift card  Lamp   Mug  Notebook   Pen  Plant  Stapler  Teapot
product
Gift card       1.00  0.60  0.53      0.43  0.48   0.53     0.43    0.52
Lamp            0.60  1.00  0.33      0.28  0.27   0.63     0.30    0.31
Mug             0.53  0.33  1.00      0.27  0.29   0.34     0.24    0.61
Notebook        0.43  0.28  0.27      1.00  0.51   0.26     0.55    0.30
Pen             0.48  0.27  0.29      0.51  1.00   0.24     0.51    0.29
Plant           0.53  0.63  0.34      0.26  0.24   1.00     0.24    0.26
Stapler         0.43  0.30  0.24      0.55  0.51   0.24     1.00    0.26
Teapot          0.52  0.31  0.61      0.30  0.29   0.26     0.26    1.00
Gift card -> Lamp 0.595
Lamp -> Plant 0.626
Mug -> Teapot 0.61
Notebook -> Stapler 0.553
Pen -> Stapler 0.506
Plant -> Lamp 0.626
Stapler -> Notebook 0.553
Teapot -> Mug 0.61

Your task: below matrix, compare the products with cosine_similarity, keep the result with product names in similarity and print it rounded to 2 digits. Then loop over the products and print each one's closest other product with its score rounded to 3 digits.

Assignment
Follow the Coding Tutorial to build the product similarity table and print every product's closest match.

Hint
cosine_similarity compares rows, so give it matrix.T. Every product scores 1.00 with itself: drop it from its own column before idxmax(), or every product picks itself.