Writing unaligned partitions creates dozens of tiny files across every output directory.
.repartition(4, 'region') on dfs['sales'] sorted by sale_id.overwrite mode partitioned by 'region' to '/output/partitioned_sales'.sale_id, region, amount) sorted by sale_id.These DataFrames are passed to solve(spark, dfs). Access one with dfs['name'].