Append and Versions

A daily job adds each day's orders to the same table. write_deltalake(folder, df, mode="append") keeps the rows already in the table and adds the DataFrame's rows. With no mode, the write stops with an error when the table already exists, so a job can't replace or grow a table by accident: you say which one you mean.

Every write makes a new version of the table: 0 for the first, then 1, 2, and so on. A version is the whole table right after that write, not just that write's rows. DeltaTable(folder) opens the newest version: dt.version() gives its number and dt.to_pandas() reads its rows into a pandas DataFrame.

dt.history() is a list with one dict per version, newest first. h["version"] and h["operation"] say which version it is and what kind of change made it; other keys hold the time of the write, which changes on every run, so don't print them. When a number looks wrong one morning, the history tells you which write changed the table.

write_deltalake("visits_delta", monday)
write_deltalake("visits_delta", tuesday, mode="append")
print(DeltaTable("visits_delta").version())   # 1

Your program appends 05-02 (12 orders) and 05-04 (9 orders) to the table that holds 05-01 (10 orders), then prints the version, the number of rows and the history. A hidden check prints the range of order ids. The output must be:

Version: 2
Rows: 31
2 WRITE
1 WRITE
0 WRITE
Order ids: 5001 to 5042 - 31 different

Your task: the starting code reads the three exports into day1, day2 and day4, writes day1 as the table orders_delta, and opens the table as dt below the writes. Append day2 and day4 with mode="append", then print "Version:" and dt.version(), "Rows:" and the number of rows, and, for each entry h of dt.history(), its version and its operation.

Assignment
Follow the Coding Tutorial to append two days to the table and read its versions.

Hint
The appends go right below the first write, above dt = DeltaTable("orders_delta"), so dt opens the newest version. len(dt.to_pandas()) counts the rows. In the history an append is a WRITE: append is the write's mode.