Getting on a hook or PostgreSQL extensibility
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Kondratov, Alexey
PostgreSQL is well-known for its extensibility. One can create their own types, operators, access methods, etc. or use powerful PL/pgSQL language to write functions, extensions, and so on and so on. Everything is thoroughly documented, supported and ready to use. However, there is also a very intimate extensibility layer, which allows external developers to peek right into the PostgreSQL core — hooks. For some reason this topic is not covered enough by official documentation. First, this talk will focus on which core hooks do exist, which options they provide for potential developers, and which PostgreSQL extensions get use of them to achieve an additional out-of-core functionality.
 Hi, my name is Alexei and I am software engineer at PostgreSQL Professional Company. And today I want to talk about specific aspects of PostgreSQL Extensibility. PostgreSQL is actually well known for its extensibility. One can create their own types, operators, access methods, or use powerful PLPG SQL language to write extensions, functions, triggers, and so on and so on. Everything is documented, supported, and ready to use. There is a lot of information in the official documentation and if we refer to it, we can even find roots of such remarkable extensibility. It says that PostgreSQL is extensible because its operation is catalogue driven. And also PostgreSQL keeps a lot of information in the catalogs and these catalogs are user-visible and it keeps not only information about databases, tables, colon names, it also raises information about operators, types, and many other things. So any user can extend PostgreSQL in many different ways. Yet, there is also a very low level extensibility layer which allows external developers to pick right into the PostgreSQL core hooks and actually callbacks that are very similar, but we will talk about them later. For some reason, this topic is not covered enough by official documentation. I think this is because it is really a low level extensibility layer and you can't use them properly without taking a look on the PostgreSQL code base. So what is a hook? Hook is just a function or actually a global pointer to a function and if it is defined, this core will execute it during standard key reprocessing or back-end lifetime at some specific moment with a predefined set of arguments. Such hooks are scattered all over the PostgreSQL core and extensions or shard libraries can set these hooks to get a view on PostgreSQL internal state or change its behavior. Let's have a look on how hooks are implemented in the code. Their implementation is very simple and straightforward. As I said before, it is just a pointer to a function and PostgreSQL checks whether it is defined or not and if it is defined, this function will be executed with some predefined set of arguments before or instead of some standard processing routine. Here is an example of hook from PostgreSQL which is called executor start. Since any hook is just a global pointer, it could be already defined and set by someone else. As that way, if extension or any external sharded library wants to install some hook, it first has to keep previous value of this pointer. After that, we can put our own pointer to our own function to this hook. This is the example of how register statements installs its hook. This model provides possibility to track planning and execution statistics of all SQL statements executed by server. That way, it has to set as many hooks as possible along the whole way of query processing. By obeying the rules that every extension should remember previous value of the hook, we build a chain of a linked list of all hooks and when hook is executed, it will also execute its predecessor and this next hook will execute its own predecessor and finally, the last hook will execute the standard processing routine. That's why every hook has to always execute its predecessor. This may be a problem because if any malicious extension will forget to call its predecessor and disables this rule, we will break standard query processing because no one will call the standard processing routine and if you lower such extension, it will break your server query processing. Every query passes through the different stages and postgres subsystems. For example, parser, analyzer, planner, and executer and finally results are returned back to the client. Hooks are placed all over the postgres query in many parts, but they are not distributed uniformly. I put some examples on the query processing flowchart and sometimes it is easy to point the hook location for executer because they are placed very straightforward in the start of the user and in many places between. All for planner where the main hook is planner hook and it's surrounded by a lot of additional hooks to add planner path and so on. But for some hooks, it is not easy to point the execution because this post-parse-analyzed hook is placed somewhere between parser and planner and we have already parsed 3 of the query but it is not planned yet. But I think it should be actually closer to the planner. As I said, hooks are not distributed uniformly and densest parts are executer, planner, and security hooks in the top of this flowchart where all the client authentication process happens. For example, this client authentication hook. All documentation prefers to keep silence about exposed hooks but there are some external sources that are well supported at least right now. First it is GitHub repository which lists all available hooks with their arguments and also every hook has a text description which says some basic information about where it is placed and how it could be used and maybe some extensions, lists all the extensions which use this hook. Next is this pgpedia website which is unofficial on documentation of the Postgres and also it has a list of hooks and the most interesting part there is that it says in which version which hook were added. And also it refers to a Git commit which introduced this hook. Also, postgresql.org website refers to these slides and talk from PgCon 2012. It is well detailed and there are a lot of information there but of course it is slightly outdated because some hooks were introduced after this talk. Let's move on to the next entity that is pretty similar to the hooks callbacks. Callbacks are mostly used for internal purposes but may be also useful for extensibility from the extensibility perspective. And some bundled with postgres extensions like postgresfdw get users there. The main difference in my understanding is that callbacks are initially designed to be set by multiple users. In other words, you don't have to care either the same callback has been already set by someone else or not like you have to keep a previous value of the hook. You just use the special set of functions to register your own function as a callback and that's all. This set of functions usually have very straightforward names like register something callback but sometimes they are not and called like before shared memory exit, on shared memory exit and a bunch of others. If the callback is defined it will be executed at some proper moment with proper arguments. Another important difference I think that callbacks provide less possibilities to modify postgres internal state and they are designed more to do a clean up for example clean error state or do something to catch invalidations and similar work. This is an example of how postgresfdw registers its callbacks to track cache invalidations and transaction state change. As I said you just have to call this set of function and it will be put into postgres grid core. Internally callback set of functions does a very similar work to what hook users do their self. It organizes a list of all registered callbacks and when it is time to go, postgres will run all of them one by one. Now we have a sense of how hooks and callbacks are implemented and work. Let's move on to an example of what an external developer can achieve with them. Nowadays everyone is talking about distributed postgres and we at postgres professional have two projects dedicated to distributing postgres over the multiple nodes. It's a multi master and shardman. This is a completely different story so let's talk about second sharded or partitioned postgres and in that case one wants to scale database horizontally. In other words we want to create partitioned tables but every node, physical node of postgres have to keep only a limited number of partitions. For example, if we're on this chart we have three nodes and table A and we split it into three partitions and first node keeps only partition one, second node keeps only partition two and third node keeps the last partition of this table. So we just distributed the data across or uniformly across this node. This is a very broad topic but let's focus on the essential part, scheme definition and creation of such partitioned tables. Of course we cannot use plain create table anymore because we have to scatter partitions across multiple nodes and also we have to link remote partitions to their physical representation because at least with postgres fw any data were accessible from any node. So client can connect to node three and also get data from partition two. So the basic way here is to just make partition two as a foreign table for partition two on the node two. So actually we need this distributed data definition language or DDL and so what requirements we can put on such distributed DDL. First we have to be able to broadcast specific or all DDL statements across a number of postgres skill nodes, second we want to create such distributed tables with familiar interface that's why we have to extend create table syntax and third is that this operation should be atomic and if you create do some DDL on multiple number of nodes this command should be either committed or aborted on all postgres skill instances at once. So we have to use so called two phase commit protocol and final requirement the most important for us is that we want to do everything from the extension and we do not want to do any core modifications here. But how does standard DDL processing work in postgres? There are a lot of steps as always but I put on this flowchart only three that matter for us. First we receive query from the client, we parse, plan, plan and do some preparation on it and finally we pass it to the processing routine which is specifically DDL. It is dedicated for processing DDL, it is called standard process utility and how we can intervene into this standard DDL processing and integrate our broadcasting routine. Actually there is a hook right in the place where we do need it, it is called process utility hook and it is executed instead of standard process utility responsible for DDL processing. And this hook receives the raw text of the statement and the plan statement. So we can decide for example if we receive create role and we decide that we want to have the same roles on every postgres instance then we can broadcast this original role statement to all postgres nodes. Code an example of this part can be found on github in our repository. Now as we have an option to broadcast an arbitrary DDL to all nodes we want to give users a convenient interface to create these distributed tables. Of course we can write some PLPJs query function, some wrapper which will do all the work and everyone can execute it on the existing table as for example CITS does and we can create a wrapper which will execute create table internally. There are some options but everyone gets used to create tables with create table statement. So we want to extend the syntax of create statements to be able to add additional parameters. In our case it would be distributed by the only one required parameter. It says that this table should be distributed across a number of nodes and it also specifies a column name which should be used for partitioning and it will be used as a partitioning key. Also we can specify a number of partitions, it may be optional if you have some default value or collocation information which means that if we have two tables with the same partitioning key and the same number of partitions we want that partitions with the same boundaries will be placed on the same physical node. So all joints executed on such partitions was local to some physical node. First let's check how posgrace reacts on unknown parameters in the create table statement. If we put this query in the psql we will see this unrecognized parameter distributed by a row and luckily it is not a syntax row which is usually thrown by the parser and it gives us a clue that parameters are not processed by the parser itself. It is not necessary because sometimes parser has some custom errors and process parameters. But still if we check the parser code we will find that it processes this unknown parameter successfully but this error arises somewhere later. So now we know that posgrace is able to parse arbitrary parameters in create table statement. However further routines do check them and do not path unrecognized parameters to the executor. And we need somehow read this additional parameters before parser verifies them and actually we are lucky enough again because we have post-parse analyzed hook which is placed after parser as I said before and we already have this parsed statement of the original query and we can check that it has this additional parameters and we can read them and keep in the local backend memory and after that remove them from this parsed statement and return control back to the core. And posgrace will not notice anything suspicious here and successfully proceed to the process utility hook where we can use our previously kept parameters to do some additional work. For example if we have distributed by we can add partitioning information there and so the further routines will create a partition table instead of a simple table. Or we will also can create partitions as well with the main partitioning table and we can decide to do broadcast or broadcast part of the work or anything else. So that's why we tricked posgrace extended syntax but posgrace thinks that we have not breaking anything. And final part of our extension to traditional DDL is atomicity. Without to PC transaction might end up committed on some nodes and aborted on others. It is not a consistent state and also it is not recoverable because you cannot abort already committed transaction. And we have to use so called to face commit protocol which introduces an intermediate state just called prepared and prepared transactions can be both committed later or aborted. And posgrace already has to PC infrastructure so we have to only use it when we are doing broadcast. I have really tried to make this flowchart readable but I'm not sure I have succeeded. Anyway, the main idea was that on the left part we have transaction flowchart. We begin it, we do some DDL there and since we have our standard process utility hook we can decide that we have to execute the same DDL on another node. This is placed on the right part of this flowchart. We begin this foreign transaction and execute this DDL there. So we have two transactions. Finally we decide that coordinator, this local transaction wants to commit and we proceed to commit. And we have transaction callback which is placed in the middle. And we have this pre-commit event and posgrace fdw for example uses this event to find a time so to decide that it is time to commit all foreign transaction which was open during this local transaction. And we first commit all foreign transaction and proceed to local commit. And everything is fine if we succeeded but if we fail at some point we may end in the state where foreign transaction was committed but local was supported and as I said it's not a recoverable state. So how we can integrate to face commit protocol here? If we consider only commit part of the previous flowchart then instead of doing commit of foreign transaction at pre-commit event in the transaction callback we can first prepare it and when we prepare successfully all foreign transactions we will proceed to the local commit and also in transaction callback we have a commit event. In this state when event commit event fired for transaction callback it is too late to abort local transaction so we will proceed to committing all prepared transaction on foreign service. And again if everything went fine then we just commit all foreign transactions and proceed it to committing all local transaction. But if we crashed at some point we may end or foreign server has crashed we may end in the state when some transactions were prepared but not committed yet. And in this case we have to introduce some additional process which may be called resolver and it could be a background walker or even external process written in any language which will crawl every server and if it finds this orphan prepared transaction it should find the state on the coordinator who initiated this transaction and decide whether to commit them or abort. Simple patch prototype which adds to face commit protocol into posgust fdw can be found in the fpgsql hacksmailing list. So finally we achieved all our points and we didn't touch posgust codebase and we can use our extension with the same packages, packets of posgusts and we do not need to recompile it and do anything else. If you have any questions, feedback or comments you can reach me directly and thank you for attending my talk. Thanks. So I think we are live now. So we have one question up for the year is about the Duke at parse level. What if we want to manage the statement at parse level? We are seeing the statement can be trapped after the parsing is done. So what we want to do before the parse is done? Do we have a look for that? No, unfortunately as far as I know we have only this post-parse analyzed hook which is only related to the parser and it receives already parsed tree where everything is already parsed so we can only modify this already parsed tree. So I think that actually we can do a lot of stuff there but still we are limited by the parser so if there are some invalid characteristics then the parser will throw this syntax error much earlier. And so I think that the process hooks only placed in some specific places due to historical reasons because some places were more interesting from the extensibility perspective like executor and planner. So far we have no option for that. There's another question. Any comments on cleanup of a new prepared transaction to avoid blocking vacuum? I was about to ask these things. How about the prepared transaction if they stay unused? Yeah, this is the right question. So if we do such implicit TPC then...
 Hi, my name is Alexei and I am software engineer at PostgreSQL Professional Company.  And today I want to talk about specific aspects of PostgreSQL Extensibility.  PostgreSQL is actually well known for its extensibility.  One can create their own types, operators, access methods, or use powerful PLPG SQL language  to write extensions, functions, triggers, and so on and so on.  Everything is documented, supported, and ready to use.  There is a lot of information in the official documentation and if we refer to it, we can  even find roots of such remarkable extensibility.  It says that PostgreSQL is extensible because its operation is catalogue driven.  And also PostgreSQL keeps a lot of information in the catalogs and these catalogs are user-visible  and it keeps not only information about databases, tables, colon names, it also raises information  about operators, types, and many other things.  So any user can extend PostgreSQL in many different ways.  Yet, there is also a very low level extensibility layer which allows external developers to  pick right into the PostgreSQL core hooks and actually callbacks that are very similar,  but we will talk about them later.  For some reason, this topic is not covered enough by official documentation.  I think this is because it is really a low level extensibility layer and you can't use  them properly without taking a look on the PostgreSQL code base.  So what is a hook?  Hook is just a function or actually a global pointer to a function and if it is defined,  this core will execute it during standard key reprocessing or back-end lifetime at some  specific moment with a predefined set of arguments.  Such hooks are scattered all over the PostgreSQL core and extensions or shard libraries can  set these hooks to get a view on PostgreSQL internal state or change its behavior.  Let's have a look on how hooks are implemented in the code.  Their implementation is very simple and straightforward.  As I said before, it is just a pointer to a function and PostgreSQL checks whether it  is defined or not and if it is defined, this function will be executed with some predefined  set of arguments before or instead of some standard processing routine.  Here is an example of hook from PostgreSQL which is called executor start.  Since any hook is just a global pointer, it could be already defined and set by someone  else.  As that way, if extension or any external sharded library wants to install some hook,  it first has to keep previous value of this pointer.  After that, we can put our own pointer to our own function to this hook.  This is the example of how register statements installs its hook.  This model provides possibility to track planning and execution statistics of all SQL statements  executed by server.  That way, it has to set as many hooks as possible along the whole way of query processing.  By obeying the rules that every extension should remember previous value of the hook,  we build a chain of a linked list of all hooks and when hook is executed, it will also execute  its predecessor and this next hook will execute its own predecessor and finally, the last  hook will execute the standard processing routine.  That's why every hook has to always execute its predecessor.  This may be a problem because if any malicious extension will forget to call its predecessor  and disables this rule, we will break standard query processing because no one will call  the standard processing routine and if you lower such extension, it will break your server  query processing.  Every query passes through the different stages and postgres subsystems.  For example, parser, analyzer, planner, and executer and finally results are returned  back to the client.  Hooks are placed all over the postgres query in many parts, but they are not distributed  uniformly.  I put some examples on the query processing flowchart and sometimes it is easy to point  the hook location for executer because they are placed very straightforward in the start  of the user and in many places between.  All for planner where the main hook is planner hook and it's surrounded by a lot of additional  hooks to add planner path and so on.  But for some hooks, it is not easy to point the execution because this post-parse-analyzed  hook is placed somewhere between parser and planner and we have already parsed 3 of the  query but it is not planned yet.  But I think it should be actually closer to the planner.  As I said, hooks are not distributed uniformly and densest parts are executer, planner, and  security hooks in the top of this flowchart where all the client authentication process  happens.  For example, this client authentication hook.  All documentation prefers to keep silence about exposed hooks but there are some external  sources that are well supported at least right now.  First it is GitHub repository which lists all available hooks with their arguments and  also every hook has a text description which says some basic information about where it  is placed and how it could be used and maybe some extensions, lists all the extensions which  use this hook.  Next is this pgpedia website which is unofficial on documentation of the Postgres and also  it has a list of hooks and the most interesting part there is that it says in which version  which hook were added.  And also it refers to a Git commit which introduced this hook.  Also, postgresql.org website refers to these slides and talk from PgCon 2012.  It is well detailed and there are a lot of information there but of course it is slightly  outdated because some hooks were introduced after this talk.  Let's move on to the next entity that is pretty similar to the hooks callbacks.  Callbacks are mostly used for internal purposes but may be also useful for extensibility from  the extensibility perspective.  And some bundled with postgres extensions like postgresfdw get users there.  The main difference in my understanding is that callbacks are initially designed to be  set by multiple users.  In other words, you don't have to care either the same callback has been already set by  someone else or not like you have to keep a previous value of the hook.  You just use the special set of functions to register your own function as a callback  and that's all.  This set of functions usually have very straightforward names like register something  callback but sometimes they are not and called like before shared memory exit, on shared memory  exit and a bunch of others.  If the callback is defined it will be executed at some proper moment with proper arguments.  Another important difference I think that callbacks provide less possibilities to modify  postgres internal state and they are designed more to do a clean up for example clean error  state or do something to catch invalidations and similar work.  This is an example of how postgresfdw registers its callbacks to track cache invalidations  and transaction state change.  As I said you just have to call this set of function and it will be put into postgres  grid core.  Internally callback set of functions does a very similar work to what hook users do  their self.  It organizes a list of all registered callbacks and when it is time to go, postgres will run  all of them one by one.  Now we have a sense of how hooks and callbacks are implemented and work.  Let's move on to an example of what an external developer can achieve with them.  Nowadays everyone is talking about distributed postgres and we at postgres professional  have two projects dedicated to distributing postgres over the multiple nodes.  It's a multi master and shardman.  This is a completely different story so let's talk about second sharded or partitioned postgres  and in that case one wants to scale database horizontally.  In other words we want to create partitioned tables but every node, physical node of postgres  have to keep only a limited number of partitions.  For example, if we're on this chart we have three nodes and table A and we split it into  three partitions and first node keeps only partition one, second node keeps only partition  two and third node keeps the last partition of this table.  So we just distributed the data across or uniformly across this node.  This is a very broad topic but let's focus on the essential part, scheme definition and  creation of such partitioned tables.  Of course we cannot use plain create table anymore because we have to scatter partitions  across multiple nodes and also we have to link remote partitions to their physical representation  because at least with postgres fw any data were accessible from any node.  So client can connect to node three and also get data from partition two.  So the basic way here is to just make partition two as a foreign table for partition two on  the node two.  So actually we need this distributed data definition language or DDL and so what requirements  we can put on such distributed DDL.  First we have to be able to broadcast specific or all DDL statements across a number of post gres skill nodes, second we want to create such distributed tables with familiar interface  that's why we have to extend create table syntax and third is that this operation should  be atomic and if you create do some DDL on multiple number of nodes this command should  be either committed or aborted on all postgres skill instances at once.  So we have to use so called two phase commit protocol and final requirement the most important  for us is that we want to do everything from the extension and we do not want to do any  core modifications here.  But how does standard DDL processing work in postgres?  There are a lot of steps as always but I put on this flowchart only three that matter for  us.  First we receive query from the client, we parse, plan, plan and do some preparation  on it and finally we pass it to the processing routine which is specifically DDL.  It is dedicated for processing DDL, it is called standard process utility and how we  can intervene into this standard DDL processing and integrate our broadcasting routine.  Actually there is a hook right in the place where we do need it, it is called process  utility hook and it is executed instead of standard process utility responsible for DDL  processing.  And this hook receives the raw text of the statement and the plan statement.  So we can decide for example if we receive create role and we decide that we want to  have the same roles on every postgres instance then we can broadcast this original role statement  to all postgres nodes.  Code an example of this part can be found on github in our repository.  Now as we have an option to broadcast an arbitrary DDL to all nodes we want to give users a  convenient interface to create these distributed tables.  Of course we can write some PLPJs query function, some wrapper which will do all the work and  everyone can execute it on the existing table as for example CITS does and we can create  a wrapper which will execute create table internally.  There are some options but everyone gets used to create tables with create table statement.  So we want to extend the syntax of create statements to be able to add additional parameters.  In our case it would be distributed by the only one required parameter.  It says that this table should be distributed across a number of nodes and it also specifies  a column name which should be used for partitioning and it will be used as a partitioning key.  Also we can specify a number of partitions, it may be optional if you have some default  value or collocation information which means that if we have two tables with the same partitioning  key and the same number of partitions we want that partitions with the same boundaries will  be placed on the same physical node.  So all joints executed on such partitions was local to some physical node.  First let's check how posgrace reacts on unknown parameters in the create table statement.  If we put this query in the psql we will see this unrecognized parameter distributed by  a row and luckily it is not a syntax row which is usually thrown by the parser and it gives  us a clue that parameters are not processed by the parser itself.  It is not necessary because sometimes parser has some custom errors and process parameters.  But still if we check the parser code we will find that it processes this unknown parameter  successfully but this error arises somewhere later.  So now we know that posgrace is able to parse arbitrary parameters in create table statement.  However further routines do check them and do not path unrecognized parameters to the  executor.  And we need somehow read this additional parameters before parser verifies them and actually we  are lucky enough again because we have post-parse analyzed hook which is placed after parser  as I said before and we already have this parsed statement of the original query and  we can check that it has this additional parameters and we can read them and keep in the local  backend memory and after that remove them from this parsed statement and return control  back to the core.  And posgrace will not notice anything suspicious here and successfully proceed to the process  utility hook where we can use our previously kept parameters to do some additional work.  For example if we have distributed by we can add partitioning information there and so  the further routines will create a partition table instead of a simple table.  Or we will also can create partitions as well with the main partitioning table and we can  decide to do broadcast or broadcast part of the work or anything else.  So that's why we tricked posgrace extended syntax but posgrace thinks that we have not  breaking anything.  And final part of our extension to traditional DDL is atomicity.  Without to PC transaction might end up committed on some nodes and aborted on others.  It is not a consistent state and also it is not recoverable because you cannot abort already  committed transaction.  And we have to use so called to face commit protocol which introduces an intermediate  state just called prepared and prepared transactions can be both committed later or aborted.  And posgrace already has to PC infrastructure so we have to only use it when we are doing  broadcast.  I have really tried to make this flowchart readable but I'm not sure I have succeeded.  Anyway, the main idea was that on the left part we have transaction flowchart.  We begin it, we do some DDL there and since we have our standard process utility hook  we can decide that we have to execute the same DDL on another node.  This is placed on the right part of this flowchart.  We begin this foreign transaction and execute this DDL there.  So we have two transactions.  Finally we decide that coordinator, this local transaction wants to commit and we proceed  to commit.  And we have transaction callback which is placed in the middle.  And we have this pre-commit event and posgrace fdw for example uses this event to find a  time so to decide that it is time to commit all foreign transaction which was open during  this local transaction.  And we first commit all foreign transaction and proceed to local commit.  And everything is fine if we succeeded but if we fail at some point we may end in the  state where foreign transaction was committed but local was supported and as I said it's  not a recoverable state.  So how we can integrate to face commit protocol here?  If we consider only commit part of the previous flowchart then instead of doing commit of foreign  transaction at pre-commit event in the transaction callback we can first prepare it and when  we prepare successfully all foreign transactions we will proceed to the local commit and also  in transaction callback we have a commit event.  In this state when event commit event fired for transaction callback it is too late to  abort local transaction so we will proceed to committing all prepared transaction on  foreign service.  And again if everything went fine then we just commit all foreign transactions and proceed  it to committing all local transaction.  But if we crashed at some point we may end or foreign server has crashed we may end in  the state when some transactions were prepared but not committed yet.  And in this case we have to introduce some additional process which may be called resolver  and it could be a background walker or even external process written in any language which  will crawl every server and if it finds this orphan prepared transaction it should find  the state on the coordinator who initiated this transaction and decide whether to commit  them or abort.  Simple patch prototype which adds to face commit protocol into posgust fdw can be found  in the fpgsql hacksmailing list.  So finally we achieved all our points and we didn't touch posgust codebase and we can  use our extension with the same packages, packets of posgusts and we do not need to  recompile it and do anything else.  If you have any questions, feedback or comments you can reach me directly and thank you for  attending my talk.  Thanks.  So I think we are live now.  So we have one question up for the year is about the Duke at parse level.  What if we want to manage the statement at parse level?  We are seeing the statement can be trapped after the parsing is done.  So what we want to do before the parse is done?  Do we have a look for that?  No, unfortunately as far as I know we have only this post-parse analyzed hook which is  only related to the parser and it receives already parsed tree where everything is already  parsed so we can only modify this already parsed tree.  So I think that actually we can do a lot of stuff there but still we are limited by the  parser so if there are some invalid characteristics then the parser will throw this syntax error  much earlier.  And so I think that the process hooks only placed in some specific places due to historical  reasons because some places were more interesting from the extensibility perspective like executor  and planner.  So far we have no option for that.  There's another question.  Any comments on cleanup of a new prepared transaction to avoid blocking vacuum?  I was about to ask these things.  How about the prepared transaction if they stay unused?  Yeah, this is the right question.  So if we do such implicit TPC then...
postgresql_extensibility_0_155.png postgresql_extensibility_156_455.png postgresql_extensibility_456_3119.png postgresql_extensibility_3120_4127.png postgresql_extensibility_4128_5171.png postgresql_extensibility_5172_6659.png postgresql_extensibility_6660_8123.png postgresql_extensibility_8124_10835.png postgresql_extensibility_10836_12851.png postgresql_extensibility_12852_15719.png postgresql_extensibility_15720_16271.png postgresql_extensibility_16272_16751.png postgresql_extensibility_16752_17063.png postgresql_extensibility_17064_20087.png postgresql_extensibility_20088_21899.png postgresql_extensibility_21900_23075.png postgresql_extensibility_23076_24347.png postgresql_extensibility_24348_27179.png postgresql_extensibility_27180_28631.png postgresql_extensibility_28632_31691.png postgresql_extensibility_31692_32783.png postgresql_extensibility_32784_35303.png postgresql_extensibility_35304_38867.png postgresql_extensibility_38868_39047.png postgresql_extensibility_39048_39335.png postgresql_extensibility_39336_39395.png postgresql_extensibility_39396_39443.png postgresql_extensibility_39444_39587.png postgresql_extensibility_39588_39839.png postgresql_extensibility_39840_40091.png postgresql_extensibility_40092_40151.png postgresql_extensibility_40152_40583.png postgresql_extensibility_40584_40691.png postgresql_extensibility_40692_41291.png postgresql_extensibility_41292_42587.png postgresql_extensibility_42588_42719.png postgresql_extensibility_42720_42767.png postgresql_extensibility_42768_42995.png postgresql_extensibility_42996_43343.png postgresql_extensibility_43344_43462.png