df = sc.parallelize([(1, '2016-01-06 00:04:21'), (2, '2016-05-01 12:20:00'),
    (3, '2016-08-06 00:04:21')]).toDF(['id', 'ts_'])
now = lit('2016-06-01 00:00:00').cast('timestamp')
five_months_ago = now - expr('INTERVAL 5 MONTHS')
df.withColumn('ts', unix_timestamp('ts_').cast('timestamp')).where(col('ts'
    ).between(five_months_ago, now)).withColumn('next_sunday', next_day(col
    ('ts'), 'Sun')).withColumn('diff', datediff(col('ts'), col('next_sunday')))
PySpark add a column to a DataFrame from a TimeStampType column