Parquet: Write and Read

A CSV keeps only text, so every reader has to guess the types again. Besides wrong guesses, there's a quieter loss: a column you turned into dates with pd.to_datetime comes back from a CSV copy as plain text (str), and date tools such as .dt.month stop working on it.

Parquet is a binary file format made for tables. Next to each column's values it stores the column's name and type, so reading it back needs no guessing: dates stay dates and whole numbers stay whole numbers. df.to_parquet(path, index=False) writes a DataFrame to a Parquet file; index=False leaves out pandas' row labels (the 0, 1, 2… on the left of a printed table), which aren't part of the data. pd.read_parquet(path) reads the file back into a DataFrame. An example on another table:

customers = pd.read_csv("customers.csv")
customers.to_parquet("customers.parquet", index=False)
again = pd.read_parquet("customers.parquet")

Why a data engineer needs it: a pipeline passes files from step to step and from team to team. With CSV, every reader must redo each pd.to_datetime and dtype=, and the one who forgets gets text where dates should be. A Parquet file carries its types, so everyone reads the same table.

Your program writes the shop's 1,497 order lines to orders.parquet and reads them back. It prints the dtypes that came back (datetime64[us] is pandas' date-and-time type), and the lesson's check prints the last four lines:

order_id                int64
customer_id             int64
date           datetime64[us]
product                   str
category                  str
quantity                int64
unit_price              int64
status                    str
dtype: object
rows read back: 1497
date from Parquet: datetime64[us]
date from a CSV copy: str
same table as the one written: True

Your task: below the date line, write orders to orders.parquet with to_parquet and index=False, read the file back into back with pd.read_parquet, and print back.dtypes.

Assignment
Follow the Coding Tutorial to save the orders as Parquet and check that their types come back unchanged.

Hint
to_parquet is a method of the DataFrame you save, like to_csv; pd.read_parquet, like pd.read_csv, takes the file's path.