Schema Drift: The Extra Column

Easy
Schema
Data Quality
CSV
DataFrame
Databricks
Snowflake

Schema Drift: The Extra Column

The CRM export added a column that your contract does not allow. A plain read now returns four columns, and every downstream select gets an unexpected shape.

Contract: customer_id int, name string, email string.

Read paths['customers'] with an explicit schema for exactly those three fields, then return them sorted by customer_id ascending.

:::{note} An explicit schema ignores extra columns instead of failing. That is one of the safest ways to absorb additive drift - the file may grow, your contract does not. :::

Expected output

customer_idnameemail
1001Asha Raoasha@example.com
Browser Engine: Evaluates PySpark DataFrame & SQL logic locally in Pyodide WebAssembly.

Input files

1 File
Loaded automatically

These files are loaded into this browser when you run. Their paths are handed to your code as the third argument: solve(spark, dfs, paths). Read one with spark.read.csv(paths['users'], header=True), or skip the read and use dfs['users'].

paths[customers]
csv
datasets/schema/crm_customers_extra.csv
customer_id(string)
name(string)
email(string)
referral_source(string)

Return these columns

customer_id(int)
name(string)
email(string)

Column names and order must match. Sort order is specified in the question.

Browser engine
Local files
Loading Python editor…
Click Run to try the sample cases, or Submit to check the hidden cases. First run caches the browser Python engine (~8–15s once); all code stays on your device.