<story><title>PostgreSQL 10 Released</title><url>https://www.postgresql.org/about/news/1786/</url></story><parent_chain><item><author>thom</author><text>It&amp;#x27;s 2017, and despite all this parallel stuff, query planners are still dumb as bricks. Every day I get annoyed at various obvious-to-the-eye WHERE clause pushdown opportunities, or situations where I can manually copy a view to a table, add indexes, and build a query on top of that quicker than letting the database fill up the entire disk with temporary files trying to do it as a subquery. It&amp;#x27;s utterly maddening if you spend your life on ad-hoc analytics&amp;#x2F;reporting workloads.&lt;p&gt;I strongly believe there are huge opportunities for AI to come along and, given a logical description of the data and query, do a better job of physically organising and serving the data. I realise some database management systems do this to an extent, but it&amp;#x27;s pretty weak sauce in my experience.</text></item></parent_chain><comment><author>anarazel</author><text>I&amp;#x27;d like to see your proposal get a bit more concrete. Your request seems a bit more for &amp;quot;magic&amp;quot; than &amp;quot;AI&amp;quot;.&lt;p&gt;You&amp;#x27;re not likely to succeed with &amp;quot;here&amp;#x27;s the query and schema&amp;quot; and have AI somehow figure out what the best plan there is. There&amp;#x27;s a lot of rules to observe for correctness, and you can&amp;#x27;t just try to execute arbitrary valid plans, because the initial query plans are going to be really bad. So you again need a model of what valid query plans are, and a cost model to evaluate how efficient a plan is, without having executed it.&lt;p&gt;I think there&amp;#x27;s plenty opportunities for ML type improvements, but they&amp;#x27;re in individual parts, less as a wholesale query planner replacement.  I think the most likely beneficiaries are around statics collection and evaluation, incremental improvements to query plans, and some execution time -&amp;gt; plan time feedback loops.</text></comment>
<story><title>PostgreSQL 10 Released</title><url>https://www.postgresql.org/about/news/1786/</url></story><parent_chain><item><author>thom</author><text>It&amp;#x27;s 2017, and despite all this parallel stuff, query planners are still dumb as bricks. Every day I get annoyed at various obvious-to-the-eye WHERE clause pushdown opportunities, or situations where I can manually copy a view to a table, add indexes, and build a query on top of that quicker than letting the database fill up the entire disk with temporary files trying to do it as a subquery. It&amp;#x27;s utterly maddening if you spend your life on ad-hoc analytics&amp;#x2F;reporting workloads.&lt;p&gt;I strongly believe there are huge opportunities for AI to come along and, given a logical description of the data and query, do a better job of physically organising and serving the data. I realise some database management systems do this to an extent, but it&amp;#x27;s pretty weak sauce in my experience.</text></item></parent_chain><comment><author>fatbird</author><text>This is interesting.  When you see these &amp;#x27;obvious to the eye&amp;#x27; opportunities, do you stop and treat them as an exercise in generalizing the optimization for a query planner?&lt;p&gt;I&amp;#x27;m curious how much query planners have to make tradeoffs between effective optimizations and not overloading the analysis phase.</text></comment>