The file datasets/json/attendance_logs.json holds one object per attendance
record - an employee can have several records for the same work_date. How many
different days did every employee show up?
paths['attendance_logs'].employee_id and count the distinct work_date values.work_days descending, then by employee_id ascending.employee_id, work_days.File questions receive the path as a third argument:
from pyspark.sql import functions as F
def solve(spark, dfs, paths):
logs = spark.read.json(paths['attendance_logs'])
# TODO: aggregate, filter and sort
return logs
The same data is also preloaded as dfs['attendance_logs'] if you prefer to skip the
file read.
This question runs in your browser - no sign-in needed. Press Run for the sample data and Submit for the hidden data.
These files are loaded into this browser when you run. Their paths are handed to your code as the third argument: solve(spark, dfs, paths). Read one with spark.read.csv(paths['users'], header=True), or skip the read and use dfs['users'].
attendance_logs]Column names and order must match. Sort order is specified in the question.