Split Documents into Chunks

Retrieval puts into the prompt only the pieces of your documents that answer the question. So first you cut every document into small pieces called chunks. A paragraph is a good size: big enough to say one whole thing, small enough that several fit in one prompt.

In a text file, a paragraph ends at a blank line, which is two newlines in a row: "\n\n" in Python. Splitting the text there gives one part per paragraph:

note = "# Opening Hours\n\nWeekdays: open at 9.\n\nSunday: closed."
parts = note.split("\n\n")
print(parts)   # ['# Opening Hours', 'Weekdays: open at 9.', 'Sunday: closed.']

Every shop document starts with a title line, like # Warranty. The title is not a chunk of its own. Instead it goes in front of each paragraph, so a chunk still makes sense when it is read alone. Each chunk is a dictionary that also records where it came from: its "source" file and its "index" in that file, counted from 0.

Run on the shop's docs/ folder, your program must print each file's name with its number of chunks, then the first chunk:

lamp.md 2
mug.md 2
payments.md 3
privacy.md 3
returns.md 5
reviews.md 3
shipping.md 4
teapot.md 2
warranty.md 3
{'source': 'lamp.md', 'index': 0, 'text': 'Desk Lamp (LP-310): An LED desk lamp with three brightness levels and a USB charging port. Price: $29.'}

The check then counts the chunks in your list: 27 in all. The last one is chunk 2 of warranty.md.

Your task: the loop that reads each .md file of docs/ is given. Write split_document(source, text), which returns the chunks of one document. Then use it in the loop: print each file's name and chunk count, gather every chunk in chunks, and print the first one.

Assignment
Follow the Coding Tutorial to write split_document and gather the chunks of every document.

Hint
parts[1:] skips the title. Read len(chunks) just before you append a chunk: that is its index. To add one file's chunks to the big list, use extend, not append.