Q:

Dose in Apache spark 1.2.1 Standalone cluster, 'number of executors equals to number of SPARK_WORKER_INSTANCES'?

Dose in Apache spark 1.2.1 Standalone cluster, 'number of executors equals to the number of SPARK_WORKER_INSTANCES' ?
I have done below setting in conf/spark-env.sh

SPARK_EXECUTOR_CORES=4
SPARK_NUM_EXECUTORS=3
SPARK_EXECUTOR_MEMORY=2G

If not can anyone tell me how to increase number of executors in standalone cluster?
Thanks in advance.

A:

In standalone mode, total executor num equals to the total worker num in your cluster.
SPARK_WORKER_INSTANCES=2 in spark-env.sh represent you want to have 2 workers process on each slave. Suppose you have n slaves you would have 2n workers, therefore, 2n executors for each job.
SPARK_NUM_EXECUTORS is not a valid setting.

