I am doing some Data Preparation on a DataFrame.Since data frames in spark are immutable, is it best practice to always create a new df-object likedf_A= df_A.read.json(jsonpath) df_A_concatenated = df_A.withColumn('colAB', concat_ws(' ', df_A.colA, df_A.colB))or just keep the name likedf_A= df_A.read.json(jsonpath) df_A = df_A.withColumn('colAB', concat_ws(' ', df_A.colA, df_A.colB))?
Good practices on working with DataFrames in Spark
data mining;spark