<story><title>Databricks response to Snowflake&apos;s accusation of lacking integrity</title><url>https://databricks.com/blog/2021/11/15/snowflake-claims-similar-price-performance-to-databricks-but-not-so-fast.html</url></story><parent_chain><item><author>drej</author><text>What I find hilarious is that companies argue who can query 100 TB faster and try to sell this to people. I&amp;#x27;ve been on the receiving end of offers by both of the companies in question and used both platforms (and sadly migrated some data jobs to them).&lt;p&gt;While they can crunch large datasets, they are laughably slow for the datasets most people have. So while I did propose we use these solutions for our big-ish data projects, management kept pushing for us to migrate our tiny datasets (tens of gigabytes or smaller) and the perf expectedly tanked compared to our other solutions (Postgres, Redshift, pandas etc.), never mind the immense costs to migrate everything and train everyone up.&lt;p&gt;Yes, these are very good products. But PLEASE, for the love of god, don&amp;#x27;t migrate to them unless you know you need them (and by &amp;#x27;need&amp;#x27; I don&amp;#x27;t mean pimping your resume).</text></item></parent_chain><comment><author>sanketsarang</author><text>I did work on making a database myself, and I must say that querying 100TB fast, let alone storing 100TB of data, is a real problem. Some companies (very few) don&amp;#x27;t have much choice but to use a DB that works on 100TB. If you do have small data, then you have a lot of options. But if your data is large, then you have very few options. So it is correct to be competing on how fast a DB can query 100TB of data; while at the same time being slow if you have just 10GB of data. Some databases are designed only for large data, and should not be used if your data is small.</text></comment>
<story><title>Databricks response to Snowflake&apos;s accusation of lacking integrity</title><url>https://databricks.com/blog/2021/11/15/snowflake-claims-similar-price-performance-to-databricks-but-not-so-fast.html</url></story><parent_chain><item><author>drej</author><text>What I find hilarious is that companies argue who can query 100 TB faster and try to sell this to people. I&amp;#x27;ve been on the receiving end of offers by both of the companies in question and used both platforms (and sadly migrated some data jobs to them).&lt;p&gt;While they can crunch large datasets, they are laughably slow for the datasets most people have. So while I did propose we use these solutions for our big-ish data projects, management kept pushing for us to migrate our tiny datasets (tens of gigabytes or smaller) and the perf expectedly tanked compared to our other solutions (Postgres, Redshift, pandas etc.), never mind the immense costs to migrate everything and train everyone up.&lt;p&gt;Yes, these are very good products. But PLEASE, for the love of god, don&amp;#x27;t migrate to them unless you know you need them (and by &amp;#x27;need&amp;#x27; I don&amp;#x27;t mean pimping your resume).</text></item></parent_chain><comment><author>StephenJGL</author><text>Very true. You have to understand the actual capabilities and your actual requirements. We work with petabyte size datasets and BigQuery is hard to beat. Our other reporting systems are still all in MySQL though.</text></comment>