Two monthly extracts of the same table arrive with the columns in different orders. A positional union would silently mix amount into customer_id.
Read paths['january'] and paths['february'] as plain header CSV (all strings), project each one onto the canonical order order_id, customer_id, amount, cast the values, and return the combined table sorted by order_id.
A positional union is the trap: Spark matches columns by position for
union, by name forunionByName.
These files are loaded into this browser when you run. Their paths are handed to your code as the third argument: solve(spark, dfs, paths). Read one with spark.read.csv(paths['users'], header=True), or skip the read and use dfs['users'].
january]february]