<story><title>One SQL to Rule Them All: Management of Streams and Tables</title><url>https://arxiv.org/abs/1905.12133</url></story><parent_chain></parent_chain><comment><author>truth_seeker</author><text>Great write-up. SQL is the most underrated language.&lt;p&gt;I am doing most of the stuff mentioned in that paper right now in my project with PipelineDB extension.&lt;p&gt;PostgreSQL 11 +&lt;p&gt;CitusDB extension (horizontal scaling and sharding) +&lt;p&gt;cstore_fdw extnesion (columnar data storage) +&lt;p&gt;file_fdw (to treat file as table) +&lt;p&gt;PipelineDB extension (Streaming computations and MV) +&lt;p&gt;TimescaleDB extension (timeseries data storage)&lt;p&gt;Also, PG 12 release will have many optimizations for table partitioning, query planning and its parallel execution</text></comment>
<story><title>One SQL to Rule Them All: Management of Streams and Tables</title><url>https://arxiv.org/abs/1905.12133</url></story><parent_chain></parent_chain><comment><author>dwenzek</author><text>A really interesting proposal to unify stream processing and relational querying and to extend SQL with a semantics over time-varying relations.&lt;p&gt;In particular, the authors (from the Apache Beam&amp;#x2F;Flink&amp;#x2F;Calcite projects) stress on the need to:&lt;p&gt;* smooth the frontier between streams and tables by using a single concept of time-varying relations;&lt;p&gt;* make explicit the difference between the event and processing times, using monotonic watermarks to tie the two;&lt;p&gt;* control when these continuously varying relations are materialized, avoiding unnecessary work and producing only timely outcomes.&lt;p&gt;These ideas are not new per se, but there are here pushed further and nicely combined.&lt;p&gt;However, I&amp;#x27;m wondering if SQL is the right tool for the task. For instance, the listing 2 seems complex when compared to the query expressed in plain English. In particular, I disagree that &amp;quot;anyone who understands enough SQL to solve a problem in a non-streaming context still has the knowledge required to solve the problem in a streaming context as well.&lt;p&gt;A must read paper by who is keen on stream processing, Spark, Flink, Beam, Kafka Stream ...</text></comment>