<story><title>EvalAI: An Open-Source Alternative to Kaggle</title><url>https://eval.ai/</url></story><parent_chain></parent_chain><comment><author>thadk</author><text>This article &amp;quot;50 Years of Data Science&amp;quot; by David Donoho (2017) had a sharp point about Kaggle-like competitions being under-valued by stats:&lt;p&gt;&lt;pre&gt;&lt;code&gt;  &amp;quot;To my mind, the crucial but unappreciated methodology driving predictive modeling’s success is what computational linguist Mark Liberman (Liberman 2010) has called the Common Task Framework (CTF). An instance of the CTF has these ingredients:
  (a) A publicly available training dataset involving, for each observation, a list of (possibly many) feature measure- ments, and a class label for that observation.
  (b) A set of enrolled competitors whose common task is to infer a class prediction rule from the training data.
  (c) A scoring referee,to which competitors can submit their prediction rule. The referee runs the prediction rule against a testing dataset, which is sequestered behind a Chinese wall. The referee objectively and automatically reports the score (prediction accuracy) achieved by the submitted rule.
  ...
  The general experience with CTF was summarized by Liberman as follows:
  1. Error rates decline by a fixed percentage each year, to an asymptote depending on task and data quality.
  2. Progress usually comes from many small improvements; a change of 1% can be a reason to break out the champagne.
  3. Shared data plays a crucial role—and is reused in unex- pected ways.
  ...

  The author believes that the Common Task Framework is the single idea from machine learning and data science that is most lacking attention in today’s statistical training.
&lt;/code&gt;&lt;/pre&gt;
&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.tandfonline.com&amp;#x2F;doi&amp;#x2F;full&amp;#x2F;10.1080&amp;#x2F;10618600.2017.1384734&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.tandfonline.com&amp;#x2F;doi&amp;#x2F;full&amp;#x2F;10.1080&amp;#x2F;10618600.2017.1...&lt;/a&gt;</text></comment>
<story><title>EvalAI: An Open-Source Alternative to Kaggle</title><url>https://eval.ai/</url></story><parent_chain></parent_chain><comment><author>tmabraham</author><text>To me this looks more like a competitor to CodaLab than Kaggle, and seems to target the ML research competition market. It seems unlike Kaggle, it doesn&amp;#x27;t seem to support things like notebooks, discussions, etc.&lt;p&gt;I might add that the comparison to Kaggle is added by the OP, I don&amp;#x27;t see it mentioned on the website anywhere.</text></comment>