Read orders_topic, filter to placed orders, then write the result to
orders_placed_topic using writeStream with format('kafka'),
outputMode('append'), and a processing-time trigger:
df.writeStream.format('kafka') .option('topic', 'orders_placed_topic') .option('startingOffsets', 'earliest') .outputMode('append') .trigger(processingTime='10 seconds') .start()
Return the filtered DataFrame. The write intent is recorded - nothing is actually sent to Kafka.
These DataFrames are passed to solve(spark, dfs). Access one with dfs['name'].