Q:

Why is my Spark Job performs two database reads and how can I avoid it

My data workflow is:
rawDf -> modifiedDf -> rollUpDf -> union(modifiedDf, rollUpDf) -> save
The performance was not good enough.
I found two database read actions, one in stage 60(generate rollUpDf) and one in stage 61. I don't understand why it needs to read the database twice as both modifiedDf and rollUpDf are from the same source.

A:

one way you can improve performance by using rawDf.cache() to retrieve the data from data base only once and then modify the data frame and roll up the data frame. This is help you to avoid reading the data twice from data base. 
Reference: (Why) do we need to call cache or persist on a RDD

