1. A method of using bi-predictive merge mode temporal prediction candidates for improving coding efficiency in decoding video data, the method comprising: receiving one or more syntax elements for a current video block, wherein the one or more syntax elements define one or more merge mode candidates from a set of candidates to be used to code the current video block according to a bi-predictive merge mode that includes a bi-predictive merge mode candidate for improving the coding efficiency; generating the set of candidates that includes the bi-predictive merge mode candidate for improving the coding efficiency, wherein a first candidate in the set of candidates is a left candidate that inherits motion information from a left neighboring block of a group of spatially neighboring video blocks of the current video block that includes multiple different spatially neighboring video blocks of the current video block that are positioned in a same frame of the encoded video data, a second candidate in the set of candidates is a temporal candidate that inherits motion information from a temporal neighbor block of the current video block that is positioned in a different frame with respect to the current video block, and a third candidate in the set of candidates is the bi-predictive merge mode candidate that inherits motion information from two different spatially neighboring video blocks of the group of spatially neighboring blocks for improving the coding efficiency, both of the two different spatially neighboring blocks being coded in uni-predictive mode; determining that the one or more syntax elements indicate an index that defines the third candidate as being selected from the set of candidates for decoding the current video block; and decoding the current video block based on the bi-predictive merge mode candidate that inherits the motion information from the two different spatially neighboring video blocks that are both coded in the uni-predictive modes.