Streaming: Watermarked Tumbling Window Aggregation

Hard
Streaming
Watermark
Aggregation
Optimization
Databricks
Uber
Confluent

Streaming: Watermarked Tumbling Window Aggregation

Register a streaming watermark on dfs['events'] using .withWatermark('event_time', '10 minutes') so Spark bounds late-event state.

Group events into 1-hour window_hour buckets (yyyy-MM-dd HH:00:00 via F.concat(F.substring(F.col('event_time'), 1, 13), F.lit(':00:00'))) and region, compute total_amount rounded to 2 decimals, and return window_hour, region, total_amount sorted by window_hour, region.

Browser Engine: Evaluates PySpark DataFrame & SQL logic locally in Pyodide WebAssembly.

Input data

0 DataFrames

These DataFrames are passed to solve(spark, dfs). Access one with dfs['name'].

Browser engine
Loading Python editor…
Click Run to try the sample cases, or Submit to check the hidden cases. First run caches the browser Python engine (~8–15s once); all code stays on your device.