Q:

pyspark merge/join two data frame

I want to merge two data frame.
list 1:
      id        
0   1234
1   456
2   789
3   101112

list 2: 
      id    age        
0   456      18
1   101112   52

Desired result:
      id    age     
0   1234     Na
1   456      18
2   789      Na
3   101112   52

tried:
list1.join(list2,list1.id==list2.id,'inner')

error:
ValueError: Can only compare identically-labeled Series objects

tried:
list1.merge(list2)

error:
result is list 2.
what is the problem?

A:

You need to perform a left_outer join so it doesn't drop records
list3 = list1.join(list2, 'id', 'left_outer')
list3.show()

+------+----+
|    id| age|
+------+----+
|   789|null|
|  1234|null|
|101112|  52|
|   456|  18|
+------+----+

