Q:

Spark - Sum values of two key-value pair RDDs

I have two files A and B whose contents are as following:
A
brown i like
big is house
jumps over lazy

B
this is my house
my house is brown
brown is color

I want to count the occurance of each word in each file separately and and then sum the results of so as to obtain count of all words in the two files i.e if a word occurs in both files then its final count would be the some total of its count in both files seprerately.
Following is the code I have written thus far:
val readme = sc.textFile("A.txt")
val readmesplit = readme.flatMap(line => line.split(" "))
val changes = sc.textFile("B.txt")
val changessplit = changes.flatMap(line => line.split(" "))

val readmeKV = readmesplit.map(x => (x, 1)).reduceByKey((x, y) => x + y)
val changesKV = changessplit.map(x => (x,1)).reduceByKey((x, y) => x + y)

val ans = readmeKV.fullOuterJoin(changesKV).collect()

This code gives me the following output:
(this,(Some(1),None)), (is,(Some(3),Some(1))), (big,(None,Some(1))),
(lazy,(None,Some(1))), (house,(Some(2),Some(1))), (over,(None,Some(1)))...and so on

Now how do I sum the value tuple of each key to obtain the occurance of each word in both files.

A:

val totals = ans.map {
  case (word, (one, two)) => (word, one.getOrElse(0) + two.getOrElse(0))
}

Just extract the two values, returning 0 if the word's not present, and add the result.

