from pyspark.sql import SparkSession
spark = SparkSession.builder.master('yarn').appName('my app'
    ).enableHiveSupport().getOrCreate()
Spark can access Hive table from pyspark but not from spark-submit