Start learning free

LESSON 42 OF 107 · DATA ENGINEERING: DATA QUALITY

Orphan Rows

Practice

An orphan row points at a parent row that doesn't exist: here, an order whose customer_id is in no row of customers.csv. It happens when an id is mistyped, or when an order arrives before its new customer does. Nothing errors, but every report that joins orders to customers (revenue per city, per age) quietly drops those orders, and its totals no longer match the shop's revenue.

A LEFT JOIN keeps every row of the left table; when a row matches nothing on the right, the right side's columns are NULL. Keeping only those rows, with a right-side column IS NULL in the WHERE, is called an anti-join. For example, the customers who never ordered:

SELECT COUNT(*)
FROM 'customers.csv' AS c
LEFT JOIN 'orders.csv' AS o
ON o.customer_id = c.customer_id
WHERE o.order_id IS NULL

A blank customer_id matches nobody either, but the not-null rule already counts it. So this rule leaves out the orders whose own id is blank, and each problem fails one check only. The order's customer_id is text and the file's is a number; DuckDB compares them for you.

A hidden step adds orders/2026-05-08.csv with two orders for customer 999, who isn't in customers.csv:

5076,999,Lamp,1,29,paid,2026-05-08 09:12:00
5077,999,Gift card,2,50,paid,2026-05-08 11:30:00

Your program prints the first line below; the check prints the rest. Paid revenue through an inner join to customers.csv is 131 lower than in raw_orders: 129 of it is these two orders, the other 2 the order with a blank customer_id. Once customer 999's orders are deleted, the rule passes.

customer_id is in customers.csv: FAIL (2)
Paid revenue in raw_orders: 2413
Paid revenue per city, added up: 2282
Without customer 999's orders:
customer_id is in customers.csv: PASS

Your task: finish the query so it counts the orders whose customer_id isn't blank and matches no customer in customers.csv, and report that count as "customer_id is in customers.csv".


Assignment
Follow the Coding Tutorial to find the orphan orders with a LEFT JOIN and an anti-join.


Hint
After the LEFT JOIN, the customer side, c.customer_id, is NULL exactly when no customer matched. The order side, o.customer_id, tells you whether the order had an id at all.


STEP 1 OF 5
    Ctrl + Enter to check