Start learning free

LESSON 2 OF 107 · DATA ENGINEERING: FILES AND FORMATS

JSON Lines

Practice

JSON Lines (files ending in .jsonl) holds one complete JSON object per line. There are no brackets around the whole file and no commas between lines: each line stands on its own. Website clicks, server logs and many exports come in this format.

To read one in plain Python, loop over the open file with for line in f: each line is a string, and json.loads turns a string of JSON into the Python value it holds, here a dict. To add a record, open the file with the mode "a" (append): writing starts after the last line, and nothing already there changes (the mode "w" would empty the file first). json.dumps does the reverse of json.loads: it turns a dict into one line of JSON text, and "\n" ends that line. An example on another file:

with open("clicks.jsonl", "a") as f:
    f.write(json.dumps({"page": "/cart", "user": 7}) + "\n")

Why a data engineer needs it: to add one record to a file that holds a single JSON list, [ … ], you must read and rewrite the whole file, because the closing ] has to stay last. A JSON Lines file only grows at the end, a reader can take it one line at a time without loading it all, and a broken line spoils only that line. pd.read_json(path, lines=True) reads a whole JSON Lines file into a DataFrame: one row per line, one column per key.

Before your code runs, the lesson writes events.jsonl fresh, so every run starts from these 10 events:

{"event": "view", "customer_id": 38, "product": "Mug", "time": "2026-05-04 09:12:05"}
{"event": "view", "customer_id": 38, "product": "Teapot", "time": "2026-05-04 09:13:40"}
{"event": "add_to_cart", "customer_id": 38, "product": "Teapot", "time": "2026-05-04 09:14:02"}
{"event": "purchase", "customer_id": 38, "product": "Teapot", "time": "2026-05-04 09:16:30"}
{"event": "view", "customer_id": 65, "product": "Lamp", "time": "2026-05-04 09:20:11"}
{"event": "view", "customer_id": 272, "product": "Plant", "time": "2026-05-04 09:31:47"}
{"event": "add_to_cart", "customer_id": 272, "product": "Plant", "time": "2026-05-04 09:32:15"}
{"event": "view", "customer_id": 65, "product": "Pen", "time": "2026-05-04 09:40:02"}
{"event": "add_to_cart", "customer_id": 65, "product": "Lamp", "time": "2026-05-04 09:41:30"}
{"event": "view", "customer_id": 84, "product": "Gift card", "time": "2026-05-04 09:55:58"}

Your code reads them, appends new_event (customer 65 buying a Lamp) as an 11th line, and reads the file with pandas. The lesson's check prints:

lines read with json.loads: 10
first event: {'event': 'view', 'customer_id': 38, 'product': 'Mug', 'time': '2026-05-04 09:12:05'}
its product: Mug
lines in the file now: 11
the file ends with a line break: True
its last line: {"event": "purchase", "customer_id": 65, "product": "Lamp", "time": "2026-05-04 10:02:41"}
rows read with pandas: 11
columns: ['event', 'customer_id', 'product', 'time']
customer_id type: int64
events by type: {'add_to_cart': 3, 'purchase': 2, 'view': 6}

Your task: fill the list events with one dict per line of events.jsonl using json.loads, append new_event to the file as one more line with json.dumps, then read the whole file into df with pd.read_json and lines=True.


Assignment
Follow the Coding Tutorial to read a JSON Lines file line by line, append an event to it and read it with pandas.


Hint
The second with starts at the left edge, below the loop, so the file you read is closed before you write to it. Without the "\n", the next event would land on the same line.


STEP 1 OF 7
    Ctrl + Enter to check