<story><title>Using ClickHouse to scale an events engine</title><url>https://github.com/getlago/lago/wiki/Using-Clickhouse-to-scale-an-events-engine</url></story><parent_chain><item><author>joshstrange</author><text>It&amp;#x27;s to power all our analytics. We ETL data into it and some data is write-once so we don&amp;#x27;t have updates&amp;#x2F;deletes but a number of our tables have summary data ETL&amp;#x27;d into them which means cleaning up the old rows.&lt;p&gt;I&amp;#x27;m sure CH shines for insert-only workloads but that doesn&amp;#x27;t cover all our needs.</text></item><item><author>nsguy</author><text>What is your use case? If you&amp;#x27;re deleting rows that already feels like maybe it&amp;#x27;s not the intended use case. I think about clickhouse as taking in a firehose of immutable data that you want to aggregate&amp;#x2F;analyze&amp;#x2F;report on. Let&amp;#x27;s say a million records per second. I&amp;#x27;ll make up an example, the orientation, speed and acceleration of every Tesla vehicle in the world in real time every second.</text></item><item><author>joshstrange</author><text>I feel like with all the Clickhouse praise on HN that we &amp;#x2F;must&amp;#x2F; be doing something fundamentally wrong because I hate every interaction I have with Clickhouse.&lt;p&gt;* Timeouts (only 30s???) unless I used the cli client&lt;p&gt;* Cancelling rows - Just kill me, so many bugs and FINAL&amp;#x2F;PREWHERE are massive foot-guns&lt;p&gt;* Cluster just feels annoying and fragile don&amp;#x27;t forget &amp;quot;ON CLUSTER&amp;quot; or you&amp;#x27;ll have a bad time&lt;p&gt;Again, I feel like we must be doing something wrong but we are paying an arm and a leg for that &amp;quot;privilege&amp;quot;.</text></item></parent_chain><comment><author>darthShadow</author><text>You have already gotten excellent options from the other comments, but here&amp;#x27;s another one that&amp;#x27;s not been mentioned yet.&lt;p&gt;You may want to consider adjusting your partition key (if feasible) as a function of datetime so you can just drop a complete partition when required, rather than needing separate delete queries.&lt;p&gt;In my experience, it has proven to be a very quick and clean way to clear out older data.</text></comment>
<story><title>Using ClickHouse to scale an events engine</title><url>https://github.com/getlago/lago/wiki/Using-Clickhouse-to-scale-an-events-engine</url></story><parent_chain><item><author>joshstrange</author><text>It&amp;#x27;s to power all our analytics. We ETL data into it and some data is write-once so we don&amp;#x27;t have updates&amp;#x2F;deletes but a number of our tables have summary data ETL&amp;#x27;d into them which means cleaning up the old rows.&lt;p&gt;I&amp;#x27;m sure CH shines for insert-only workloads but that doesn&amp;#x27;t cover all our needs.</text></item><item><author>nsguy</author><text>What is your use case? If you&amp;#x27;re deleting rows that already feels like maybe it&amp;#x27;s not the intended use case. I think about clickhouse as taking in a firehose of immutable data that you want to aggregate&amp;#x2F;analyze&amp;#x2F;report on. Let&amp;#x27;s say a million records per second. I&amp;#x27;ll make up an example, the orientation, speed and acceleration of every Tesla vehicle in the world in real time every second.</text></item><item><author>joshstrange</author><text>I feel like with all the Clickhouse praise on HN that we &amp;#x2F;must&amp;#x2F; be doing something fundamentally wrong because I hate every interaction I have with Clickhouse.&lt;p&gt;* Timeouts (only 30s???) unless I used the cli client&lt;p&gt;* Cancelling rows - Just kill me, so many bugs and FINAL&amp;#x2F;PREWHERE are massive foot-guns&lt;p&gt;* Cluster just feels annoying and fragile don&amp;#x27;t forget &amp;quot;ON CLUSTER&amp;quot; or you&amp;#x27;ll have a bad time&lt;p&gt;Again, I feel like we must be doing something wrong but we are paying an arm and a leg for that &amp;quot;privilege&amp;quot;.</text></item></parent_chain><comment><author>nsguy</author><text>You can always use different databases for different use cases.&lt;p&gt;There are many applications that require extremely high insertion rates (millions of records per second), very large total number of rows (billions, trillions) and flexible&amp;#x2F;fast querying&amp;#x2F;aggregation with high read rates (100&amp;#x27;s of millions or higher rows&amp;#x2F;s) and that&amp;#x27;s sort of the sweet spot IMO for ClickHouse and where you&amp;#x27;ll be pressed to find alternatives. I&amp;#x27;m sure it can be used in other situations but maybe there are more choices if you&amp;#x27;re in those.</text></comment>