<story><title>Announcing Spark 1.6</title><url>https://databricks.com/blog/2016/01/04/announcing-spark-1-6.html</url></story><parent_chain></parent_chain><comment><author>mb22</author><text>We&amp;#x27;ve been testing 1.6 since before release, specifically SparkSQL and there are some big performance improvements in this release.
We&amp;#x27;re putting together a 3rd party benchmark I&amp;#x27;ll post to HN when we are done.</text></comment>
<story><title>Announcing Spark 1.6</title><url>https://databricks.com/blog/2016/01/04/announcing-spark-1-6.html</url></story><parent_chain></parent_chain><comment><author>kod</author><text>So the detailed post on datasets at &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;databricks.com&amp;#x2F;blog&amp;#x2F;2016&amp;#x2F;01&amp;#x2F;04&amp;#x2F;introducing-spark-datasets.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;databricks.com&amp;#x2F;blog&amp;#x2F;2016&amp;#x2F;01&amp;#x2F;04&amp;#x2F;introducing-spark-dat...&lt;/a&gt;&lt;p&gt;uses groupBy&lt;p&gt;I&amp;#x27;m pretty sure based on previous comments you&amp;#x27;ve made that groupBy was one of the things you&amp;#x27;d rather eliminate from the RDD api, because of the performance impact compared to reduceByKey (which is almost always what people should be using instead).&lt;p&gt;Are you at all worried about confusion if groupBy now performs ok on datasets, but not on rdds?</text></comment>