Q:

spark scala dataset union Dataset[Array[String]]

I have two Dataset[Array[String]] . 
first Dataset having records like 
8473245,0vqt141w2d6xfmsd1,c32d6691d56d27269d8d46e7e1caf950,p
11117398,0vqt141w3c57bb1b1,c32d6691d56d27269d8d46e7e1caf950,p
329726,1141556550,c0e85ebe26e6befa568502a3c7265c34,p
601438,1149608959,ab0e30f4d904d7f6dfbd48ff7ea955d5,p

another dataset having records like 
8387081,0vqt141w7h6rwq9b1,c32d6691d56d27269d8d46e7e1caf950,p
1194736,1243752711,c0e85ebe26e6befa568502a3c7265c34,p

now I want a dataset that has all the records from first dataset
and only those records from second dataset that has the 2nd field match with the first dataset.means resulting dataset should have records:
8473245,0vqt141w2d6xfmsd1,c32d6691d56d27269d8d46e7e1caf950,p
11117398,0vqt141w3c57bb1b1,c32d6691d56d27269d8d46e7e1caf950,p
329726,1141556550,c0e85ebe26e6befa568502a3c7265c34,p
601438,1149608959,ab0e30f4d904d7f6dfbd48ff7ea955d5,p
8387081,0vqt141w7h6rwq9b1,c32d6691d56d27269d8d46e7e1caf950,p

A:

Since you don't have header lines in the data I suppose you are reading data as following (I have used one 2nd column matching for test purpose)
val dataset1 = sqlContext.read.option("header", false).csv("file1 path")
val dataset2 = sqlContext.read.option("header", false).csv("file2 path")

which should give you
dataset1
+--------+-----------------+--------------------------------+---+
|_c0     |_c1              |_c2                             |_c3|
+--------+-----------------+--------------------------------+---+
|8473245 |0vqt141w2d6xfmsd1|c32d6691d56d27269d8d46e7e1caf950|p  |
|11117398|0vqt141w3c57bb1b1|c32d6691d56d27269d8d46e7e1caf950|p  |
|329726  |1141556550       |c0e85ebe26e6befa568502a3c7265c34|p  |
|601438  |1149608959       |ab0e30f4d904d7f6dfbd48ff7ea955d5|p  |
+--------+-----------------+--------------------------------+---+
    
dataset2
+-------+-----------------+--------------------------------+---+
|_c0    |_c1              |_c2                             |_c3|
+-------+-----------------+--------------------------------+---+
|8387081|0vqt141w2d6xfmsd1|c32d6691d56d27269d8d46e7e1caf950|p  |
|1194736|1243752711       |c0e85ebe26e6befa568502a3c7265c34|p  |
+-------+-----------------+--------------------------------+---+

Following solution should work for you (commented as explanation)
//dataset2 columns to be selected
val columnsToSelect = dataset2.columns

//joining and selecting matching rows
import org.apache.spark.sql.functions._
val matchingDataset2 = dataset1.as("table1").join(dataset2.as("table2"), col("table1._c1") === col("table2._c1"))
  .select(columnsToSelect.map(x => col("table2."+x)): _*)

//merging matching dataset2 with dataset1
dataset1.union(matchingDataset2)

which should give you
+--------+-----------------+--------------------------------+---+
|_c0     |_c1              |_c2                             |_c3|
+--------+-----------------+--------------------------------+---+
|8473245 |0vqt141w2d6xfmsd1|c32d6691d56d27269d8d46e7e1caf950|p  |
|11117398|0vqt141w3c57bb1b1|c32d6691d56d27269d8d46e7e1caf950|p  |
|329726  |1141556550       |c0e85ebe26e6befa568502a3c7265c34|p  |
|601438  |1149608959       |ab0e30f4d904d7f6dfbd48ff7ea955d5|p  |
|8387081 |0vqt141w2d6xfmsd1|c32d6691d56d27269d8d46e7e1caf950|p  |
+--------+-----------------+--------------------------------+---+

I hope the answer is helpful
Updated
You have commented

Is there any way to convert org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] to Dataset[Array(String)]

You can call a map function and change the data according to your need as
dataset1.union(matchingDataset2)
  .map(row => row.toSeq.map(_.toString))
//org.apache.spark.sql.Dataset[Seq[String]] = [value: array<string>]

