An upstream job sometimes drops customer_id from orders.csv. Loading that file silently produces nulls, so the bad rows only surface days later as unattributed revenue.
Rule: inspect the header before you apply the schema. If customer_id is missing, raise a ValueError whose message contains customer_id. When the column is present, return order_id, customer_id, amount sorted by order_id.
This is a fail-fast contract: a broken file must stop the pipeline, not produce a table with nulls.
header = spark.read.option('header', True).csv(paths['orders'])
These files are loaded into this browser when you run. Their paths are handed to your code as the third argument: solve(spark, dfs, paths). Read one with spark.read.csv(paths['users'], header=True), or skip the read and use dfs['users'].
orders]