PostgreSQL Architectures in Production
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Fontaine, Dimitri
When using PostgreSQL in production it is important to implement a strategy for High Availability. With a database service, the HA properties apply to both the service itself and of course to the data set. In this talk we learn how to think about specific HA needs of your production environment and how to achieve your requirements with Open Source tooling when it comes to your database of choice, PostgreSQL. In particular, we dive in many options that could be implemented for Postgres to evolve its offering from being a toolset to being “batteries included”. What does it mean in the context of HA? How to achieve it?
 Hi and welcome to FOSDEM 2021. In the PostgreSQL developer room today we are talking about PostgreSQL architectures in production. I am Dimitri Fontaine and I have written a book named The Art of PostgreSQL. It's a book for application developers that you can find in a paperback form or that you can have online as an EPUB or PDF format of course. So this book is mainly for developers, application developers who want to be better at benefiting from PostgreSQL advanced SQL feature sets. And today in the context of FOSDEM I am offering a special discount code for them 2021. So use that in the checkout form on the website, theartofpostgreSQL.com for a 30% discount. Back to the main topic of today, PostgreSQL architectures in production. What does this title mean exactly? It's a pretty long phrase so let's zoom in and detail every word of this sentence. PostgreSQL architectures in production. The first word of course is PostgreSQL. The world's most advanced open source relational database. That's mouthful. What does it mean for your application as an application developer? The part in the stack of your application where you run PostgreSQL is going to solve an important problem for you. In a way that you don't have to think so much about it in the code you write to run your business, your application, your logic. And this problem is concurrency. You want a single data set managed in PostgreSQL in such a way that many concurrent transactions may happen at the same time in a way that your business guarantees are indeed guaranteed by the system by PostgreSQL. So what you solve with PostgreSQL in your stack that's concurrency. Now the second word in the phrase is architectures. So if you draw your stack as an application developer with the different components as they run in production as different services that connect to each other and if you draw the dependencies between the services that's what we call a production architecture. And in production means it's live now. In the image of a picture for the slide you can see a sport event being broadcasted on television on TV. For the broadcast to happen you need production engineers to actually be there at the console while the event is happening. In our job we are lucky enough that we don't need to be there at all time while customers are using our software. We can deploy it and it mostly works automatically. So the key for that to happen is availability. In our case because we are talking about PostgreSQL which is a database server we are talking about the availability of both the service and the data. And if you don't want to be at the console while the user can actually use your application you want high availability. So the availability can be measured in a number of seconds of done time per period of time usually per year. And if you make the ratio in between the interval of time when the service was available compared to when it was not then you have the famous 99.999% of availability. If you reach a certain threshold that should make sense depending on the application and the business you are working in. Well then you have high availability. To reach high availability if you're allowed like five minutes a year of done time in the case of failure of hardware component in production well what's next. If you only have five minutes a year to react to that you want as much as possible to be automated. So how do you automate a failover the process of going from one box that has failed now to another one to host both the PostgreSQL service and data online. Can you automate all of that and how much of it should you automate. When using PostgreSQL it is the classic PostgreSQL architecture that allows you to automate some of the things. Usually we use one primary and two stand-by's in a way that even if you lose a stand-by you know you still have two copies of the data at all time which means you can still accept transactions that are modifying the dataset writes in your application. Because the main issue is if you have no stand-by at all if you are left with one node the primary should you accept writes so you can either protect the data at this point and then you need to refuse the writes or you can protect the service and then you accept the writes but in case of the next failure well then you might lose data. So that's the rule trade-off that you need to decide with a database system that needs to handle availability of both the service and the data. So the next part of this presentation is going to focus on the tool that are provided for by PostgreSQL by core Postgres for you to implement failover. So included in Postgres to implement failover is streaming replication and streaming replication is provided in a way that it can be set up to be either synchronous or asynchronous. When we say that it's a sync synchronous we don't mean to say as in the real world sync would mean that two different events are happening at different places at exactly the same time they're in sync. It's not what it means we pretend it's sync because we wait until it happened in this other place. In the drawing there we have three nodes A, B and C. If A is the primary and you commit a transaction on A then A will wait until the transaction is known to have been committed on at least B or C at least one of them by default until it reports to the client connection that the commit has happened. That's what we mean with a sync. Sync means I'm happy to wait. That's all it means in PostgreSQL streaming replication. We don't have a magic wand to go faster than the speed of light. Speed of light is still a thing so we need to wait until the event could happen on another system. What's unique in PostgreSQL with this setting is that it applies per transaction. You can have on the same system in flight at the same time transactions that are going to wait and transactions that are going to not wait and be async asynchronous. You can even as in the slide there have a connection string with a special VIP user for the database that is always going to wait because it's only doing VIP transactions and you want to make sure those transactions made it to the other servers. Included in PostgreSQL to handle failover and high availability is also online streaming changes. You can register in PostgreSQL replication slots so that you make sure that the primary at all time keeps the data that is necessary for its secondaries to continue streaming. We also include in PostgreSQL tools such as PgBasbackup and PgRewind that are useful to set up and maintain stand-byes in a failover system. So you can have changes in roles and thanks to PgRewind you can grab the missing wall and make it back to follow the new primary when that happens. There is also the idea of fast forward and PostgreSQL by default includes cascading replication that allows a standby node to fetch the missing wall from another standby node. Also PostgreSQL includes online standby promotion which is the idea that when the standby is implementing read-on-lequeries at the time of the promotion from standby to primary those queries are left to continue. So there is no stop in the work stream on the standby. That's why it's called the hot standby and it's referred to also in this slide as online standby promotion. Including PostgreSQL also is the basics of point-in-time recovery. I say the basics because for point-in-time recovery to actually be useful to actually work to implement that you need an archive, an archive of both base backups and also wall files. The wall is the right head log and that's needed for recovery purposes and also for streaming replication. The archiving is a concept in the PostgreSQL implementation. You need to provide it separately. So please use an existing solution for handling archiving such as PgBackrest, WALL-E, WALL-G, PgBarman or maybe another of those open source solutions that are part of the ecosystem of PostgreSQL. Included in PostgreSQL is also ClientSideHA. On the client side in your application code when you connect to PostgreSQL and the failover happens, well your connection will break. The client will lose its connection and when connection is lost it's up to the client to realize and then connect again. What is provided for in PostgreSQL, what is included is a connection string facility that allows you to specify many hosts and then have LIPPQ so the client side that establishes the connection for you connect to this one node among the list that is currently the primary in a way that you can reuse the same connection string again when you lose the connection on the failover and it's to reconnect. The discovery of which node is currently the primary that's included in PostgreSQL. It's the target session attributes that you can see on the slide on the connection string there. Now some of the things that you need are not included in PostgreSQL. The first that I want to list is archiving. Archiving is a concept in PostgreSQL. It's not included directly in PostgreSQL core. Again use an existing open source software that implements archiving for you. I've listed them just previously. With archiving we implement bus backup and war archiving like I said before. Also not included in PostgreSQL is this idea that nodes will have a rule that changes over time. The currently primary node is going to be a secondary sometime and one of the secondaries after a failover is going to be a primary. That dynamic idea about rules, this idea that rules are dynamic is not included in PostgreSQL. Some other systems implement raft or paxos or other such algorithms that allow to have a dynamic rule in the system. That's not the case for PostgreSQL. Nothing included in PostgreSQL is configuration management, extension management and upgrades. Configuration management is the idea that if you change PostgreSQL.com for pghba.com for some other settings that way you need to take care of deploying this change to all the nodes because otherwise when you failover the changes you just made to the primary are not automatically available on the secondary yet. That's a separate tooling that you need to put in place. Extension management also is a little tricky because when you do create extension all the SQL objects are going to be replicated on the standby. The standby still needs to have the operating system interface, the.so, the shared library that is necessary for the extension. It needs to be installed both on the primary and the secondary servers and PostgreSQL on the secondary when it does create extension. It will not verify for you that the.so is there and available. That's an operating system integration job and PostgreSQL is not going to do it for you. It's not included. Also upgrades in the case of a multi-node system. You need to upgrade nodes separately and schedule them and schedule the restarts and the reconnection of the application yourself. That's the main thing about implementing failover with PostgreSQL. Now what if we want to automate entirely the operation of a failover? It's possible to do so. There are multiple tools allowing you to do that. I have worked on a tool named Pgauto Failover that I want to present today in this talk. The default first simple setup for Pgauto Failover is shown on the slide there. It consists of one primary, one secondary, and what we call the monitor node. The monitor is a third node that allows us to notice network splits and decide if one node is unavailable just because of network issues or completed down. So we can witness that and react correctly to network split situations thanks to the monitor. In this architecture with a single standby, you can implement automated failover. What you cannot provide with that architecture as we said before is both service and data availability. With this architecture you need to choose either you want to provide the service first, that's what we do, and in that case when you lose the secondary you still accept writes on the primary. Or you can implement data security, data availability, and then refuse your application to be able to write to the PostgreSQL server when the secondary is down. That's not what we do by default in that case. So it's not proper HA there, it's just failover capacity, but maybe that's what you need. So how do you make it happen? In Pgauto Failover we have a tool named Pgauto Ctl that is part of the package. So you Pgauto Ctl create the monitor and then twice in a row you create a PostgreSQL instance. The monitor is going to register the instance and decide if it's going to be a primary or secondary depending on the order in which you type the commands. Basically the first server is going to be initialized as a primary and the second one because the primary already exists is going to be initialized as a standby as a secondary node. In Pgauto Failover we named the secondary a node that way. The secondary is a standby that you can promote when the primary fails basically. So any node could be a standby but only those standby is ready for taking over, failing over from the primary or named a secondary. So when you have already one standby node, if you want to actually implement HA maybe you want a second standby node. So how do you make it happen? You just type the exact same command again Pgauto Ctl create PostgreSQL with the same set of arguments and it's going to register your third node on a third host or VM or a bare bone machine as you wish to the monitor and then give it the rule that you expect and everything is going to be set up for you. Sometimes you want to have three standby nodes so again same command and when we do that it sometimes it's because the third node is going to be a little different from the two previous nodes. It might sit on a zone that is a remote zone and it's there only for disaster recovery purposes. You never want to automatically fail over to that node, still you want it to maintain a copy of the data. So maybe you want node C in this diagram to not participate into the replication quorum and to never be a candidate for failover. So you want its candidate priority to be zero. The way to make that happen is either you knew that Pgauto Ctl create PostgreSQL time and then you run the command with the new arguments there that you can see on the slide. Replication quorum false and dash dash candidate priority zero or maybe you didn't know when you installed it the first time and then you can run the second part of the slide that the two commands Pgauto Ctl set node candidate priority or replication quorum. You can do that online and you can change your mind many times in the system while it's running in production and this is going to dynamically apply to your already running system. So included in PostgreSQL with Pgauto failover sorry so included in Pgauto failover same way as in PostgreSQL is streaming replication with replication slot. The thing that we do that is not default in PostgreSQL is maintain the replication slot understand by nodes themselves so that after a failover everything is smoothly reconnecting and just continuing from where it was. We include also an easy to set up SSL certificate set of facilities and we include the idea that the all the settings the replication settings are dynamic you can change them online with a very simple command. So included in Pgauto failover is also the idea that roles are dynamic. A node that is a primary is going to be a secondary if it fails and the node that is a secondary might become a primary later so those roles are dynamic in Pgauto failover. You can also use some of the commands presented in the slide to figure out the current properties of the system. Also included in Pgauto failover is online membership changes just like I said before when the node fails then something happens and then the membership changes from primary to secondary or secondary to primary things like that. We also include support for maintenance operation. A node that is ongoing maintenance cannot be a candidate for failover of course. So you can enable maintenance on a node do a kernel security upgrade for example and then when it's back online you can disable maintenance for this node to be a candidate for failover again when it's back to being ready for that. Included also in Pgauto failover is network split detection and protection thanks to the monitor being there in the architecture and thanks also to some smarts included in Pgauto CTL client itself. Pgauto CTL is going to manage the post-grasqual instance and also check out some network properties and make some decisions about the situation. You can even perform a promotion and target manually the node that you want to be the new primary which might be useful if you're migrating from some data center to another or if you change your current primary availability zone that's possible to do with a simple perform promotion command in Pgauto failover. Not included yet in Pgauto failover is archiving and disaster recovery setup. The idea is that in most of the existing archiving and disaster recovery solutions the idea that roles are dynamic is not implemented. There is a walkthrough in the tutorial or in the documentation to install the solution with a known current primary. It's hard to figure out what you should do when this primary is going to change and then to make it automatic. So that's why I think I believe that we should have something about that in Pgauto failover itself. And again configuration management, extension management etc. It's not today part of what Pgauto failover is going to do on your clusters in prediction. So that's it. That's the talk. That's my presentation today post-grasqual architectures in prediction. Feel free to ask me questions now and have a good day. Thank you.
 Hi and welcome to FOSDEM 2021. In the PostgreSQL developer room today we are talking about  PostgreSQL architectures in production. I am Dimitri Fontaine and I have written a book  named The Art of PostgreSQL. It's a book for application developers that you can find  in a paperback form or that you can have online as an EPUB or PDF format of course.  So this book is mainly for developers, application developers who want to be better at benefiting  from PostgreSQL advanced SQL feature sets. And today in the context of FOSDEM I am offering a  special discount code for them 2021. So use that in the checkout form on the website,  theartofpostgreSQL.com for a 30% discount. Back to the main topic of today, PostgreSQL  architectures in production. What does this title mean exactly? It's a pretty long phrase so let's  zoom in and detail every word of this sentence. PostgreSQL architectures in production. The first  word of course is PostgreSQL. The world's most advanced open source relational database. That's  mouthful. What does it mean for your application as an application developer? The part in the stack  of your application where you run PostgreSQL is going to solve an important problem for you.  In a way that you don't have to think so much about it in the code you write to run your business,  your application, your logic. And this problem is concurrency. You want a single data set managed  in PostgreSQL in such a way that many concurrent transactions may happen at the same time in a  way that your business guarantees are indeed guaranteed by the system by PostgreSQL. So what  you solve with PostgreSQL in your stack that's concurrency. Now the second word in the phrase  is architectures. So if you draw your stack as an application developer with the different  components as they run in production as different services that connect to each other and if you  draw the dependencies between the services that's what we call a production architecture. And in  production means it's live now. In the image of a picture for the slide you can see a sport event  being broadcasted on television on TV. For the broadcast to happen you need production engineers  to actually be there at the console while the event is happening. In our job we are lucky enough  that we don't need to be there at all time while customers are using our software. We can deploy  it and it mostly works automatically. So the key for that to happen is availability. In our case  because we are talking about PostgreSQL which is a database server we are talking about the  availability of both the service and the data. And if you don't want to be at the console while  the user can actually use your application you want high availability. So the availability can be  measured in a number of seconds of done time per period of time usually per year. And if you make  the ratio in between the interval of time when the service was available compared to when it was  not then you have the famous 99.999% of availability. If you reach a certain threshold that should  make sense depending on the application and the business you are working in. Well then you have  high availability. To reach high availability if you're allowed like five minutes a year of done  time in the case of failure of hardware component in production well what's next. If you only have  five minutes a year to react to that you want as much as possible to be automated. So how do you  automate a failover the process of going from one box that has failed now to another one to  host both the PostgreSQL service and data online. Can you automate all of that and how much of it  should you automate. When using PostgreSQL it is the classic PostgreSQL architecture that allows  you to automate some of the things. Usually we use one primary and two stand-by's in a way that  even if you lose a stand-by you know you still have two copies of the data at all time which  means you can still accept transactions that are modifying the dataset writes in your application.  Because the main issue is if you have no stand-by at all if you are left with one node the  primary should you accept writes so you can either protect the data at this point and then you need  to refuse the writes or you can protect the service and then you accept the writes but in  case of the next failure well then you might lose data. So that's the rule trade-off that you need  to decide with a database system that needs to handle availability of both the service and the data.  So the next part of this presentation is going to focus on the tool that are provided for by  PostgreSQL by core Postgres for you to implement failover. So included in Postgres to implement  failover is streaming replication and streaming replication is provided in a way that it can  be set up to be either synchronous or asynchronous. When we say that it's a sync synchronous we don't  mean to say as in the real world sync would mean that two different events are happening at different  places at exactly the same time they're in sync. It's not what it means we pretend it's sync because  we wait until it happened in this other place. In the drawing there we have three nodes A, B and C.  If A is the primary and you commit a transaction on A then A will wait until the transaction  is known to have been committed on at least B or C at least one of them by default until it  reports to the client connection that the commit has happened. That's what we mean with a sync.  Sync means I'm happy to wait. That's all it means in PostgreSQL streaming replication. We don't  have a magic wand to go faster than the speed of light. Speed of light is still a thing so we  need to wait until the event could happen on another system. What's unique in PostgreSQL with  this setting is that it applies per transaction. You can have on the same system in flight at the  same time transactions that are going to wait and transactions that are going to not wait and be  async asynchronous. You can even as in the slide there have a connection string with a special VIP  user for the database that is always going to wait because it's only doing VIP transactions and you  want to make sure those transactions made it to the other servers. Included in PostgreSQL to handle  failover and high availability is also online streaming changes. You can register in PostgreSQL  replication slots so that you make sure that the primary at all time keeps the data that is necessary  for its secondaries to continue streaming. We also include in PostgreSQL tools such as PgBasbackup and  PgRewind that are useful to set up and maintain stand-byes in a failover system. So you can have  changes in roles and thanks to PgRewind you can grab the missing wall and make it back to follow  the new primary when that happens. There is also the idea of fast forward and PostgreSQL by default  includes cascading replication that allows a standby node to fetch the missing wall from another  standby node. Also PostgreSQL includes online standby promotion which is the idea that when the  standby is implementing read-on-lequeries at the time of the promotion from standby to primary  those queries are left to continue. So there is no stop in the work stream on the standby. That's  why it's called the hot standby and it's referred to also in this slide as online standby promotion.  Including PostgreSQL also is the basics of point-in-time recovery. I say the basics because  for point-in-time recovery to actually be useful to actually work to implement that you need an  archive, an archive of both base backups and also wall files. The wall is the right head log and  that's needed for recovery purposes and also for streaming replication. The archiving is a concept  in the PostgreSQL implementation. You need to provide it separately. So please use an existing  solution for handling archiving such as PgBackrest, WALL-E, WALL-G, PgBarman or maybe another of those  open source solutions that are part of the ecosystem of PostgreSQL. Included in PostgreSQL is also  ClientSideHA. On the client side in your application code when you connect to PostgreSQL and the  failover happens, well your connection will break. The client will lose its connection and when  connection is lost it's up to the client to realize and then connect again. What is provided for in  PostgreSQL, what is included is a connection string facility that allows you to specify many hosts and  then have LIPPQ so the client side that establishes the connection for you connect to this one node  among the list that is currently the primary in a way that you can reuse the same connection string  again when you lose the connection on the failover and it's to reconnect. The discovery of which node  is currently the primary that's included in PostgreSQL. It's the target session attributes that  you can see on the slide on the connection string there. Now some of the things that you need are  not included in PostgreSQL. The first that I want to list is archiving. Archiving is a concept in  PostgreSQL. It's not included directly in PostgreSQL core. Again use an existing open source software  that implements archiving for you. I've listed them just previously. With archiving we implement  bus backup and war archiving like I said before. Also not included in PostgreSQL is this idea that nodes  will have a rule that changes over time. The currently primary node is going to be a secondary  sometime and one of the secondaries after a failover is going to be a primary. That dynamic idea  about rules, this idea that rules are dynamic is not included in PostgreSQL. Some other systems  implement raft or paxos or other such algorithms that allow to have a dynamic rule in the system.  That's not the case for PostgreSQL. Nothing included in PostgreSQL is configuration management,  extension management and upgrades. Configuration management is the idea that if you change  PostgreSQL.com for pghba.com for some other settings that way you need to take care of  deploying this change to all the nodes because otherwise when you failover the changes you just  made to the primary are not automatically available on the secondary yet. That's a separate  tooling that you need to put in place. Extension management also is a little tricky because when  you do create extension all the SQL objects are going to be replicated on the standby.  The standby still needs to have the operating system interface, the.so, the shared library  that is necessary for the extension. It needs to be installed both on the primary and the  secondary servers and PostgreSQL on the secondary when it does create extension. It will not  verify for you that the.so is there and available. That's an operating system integration job and  PostgreSQL is not going to do it for you. It's not included. Also upgrades in the case of a  multi-node system. You need to upgrade nodes separately and schedule them and schedule the  restarts and the reconnection of the application yourself. That's the main thing about implementing  failover with PostgreSQL. Now what if we want to automate entirely the operation of a failover?  It's possible to do so. There are multiple tools allowing you to do that. I have worked on a tool  named Pgauto Failover that I want to present today in this talk. The default first simple setup for  Pgauto Failover is shown on the slide there. It consists of one primary, one secondary,  and what we call the monitor node. The monitor is a third node that allows us to notice network  splits and decide if one node is unavailable just because of network issues or completed down.  So we can witness that and react correctly to network split situations thanks to the monitor.  In this architecture with a single standby, you can implement automated failover. What you cannot  provide with that architecture as we said before is both service and data availability.  With this architecture you need to choose either you want to provide the service first, that's what  we do, and in that case when you lose the secondary you still accept writes on the primary. Or you  can implement data security, data availability, and then refuse your application to be able to write  to the PostgreSQL server when the secondary is down. That's not what we do by default in that case.  So it's not proper HA there, it's just failover capacity, but maybe that's what you need. So how  do you make it happen? In Pgauto Failover we have a tool named Pgauto Ctl that is part of the  package. So you Pgauto Ctl create the monitor and then twice in a row you create a PostgreSQL  instance. The monitor is going to register the instance and decide if it's going to be a primary  or secondary depending on the order in which you type the commands. Basically the first server is  going to be initialized as a primary and the second one because the primary already exists  is going to be initialized as a standby as a secondary node. In Pgauto Failover we named the  secondary a node that way. The secondary is a standby that you can promote when the primary  fails basically. So any node could be a standby but only those standby is ready for taking over,  failing over from the primary or named a secondary. So when you have already one standby node,  if you want to actually implement HA maybe you want a second standby node. So how do you make it  happen? You just type the exact same command again Pgauto Ctl create PostgreSQL with the same set of  arguments and it's going to register your third node on a third host or VM or a bare bone machine as  you wish to the monitor and then give it the rule that you expect and everything is going to be set  up for you. Sometimes you want to have three standby nodes so again same command and when we do that  it sometimes it's because the third node is going to be a little different from the two previous  nodes. It might sit on a zone that is a remote zone and it's there only for disaster recovery  purposes. You never want to automatically fail over to that node, still you want it to maintain  a copy of the data. So maybe you want node C in this diagram to not participate into the  replication quorum and to never be a candidate for failover. So you want its candidate priority  to be zero. The way to make that happen is either you knew that Pgauto Ctl create PostgreSQL time  and then you run the command with the new arguments there that you can see on the slide.  Replication quorum false and dash dash candidate priority zero or maybe you didn't know when you  installed it the first time and then you can run the second part of the slide that the two  commands Pgauto Ctl set node candidate priority or replication quorum. You can do that online  and you can change your mind many times in the system while it's running in production and this  is going to dynamically apply to your already running system. So included in PostgreSQL with  Pgauto failover sorry so included in Pgauto failover same way as in PostgreSQL is streaming  replication with replication slot. The thing that we do that is not default in PostgreSQL is  maintain the replication slot understand by nodes themselves so that after a failover everything is  smoothly reconnecting and just continuing from where it was. We include also an easy to set up  SSL certificate set of facilities and we include the idea that the all the settings the replication  settings are dynamic you can change them online with a very simple command. So included in Pgauto  failover is also the idea that roles are dynamic. A node that is a primary is going to be a secondary  if it fails and the node that is a secondary might become a primary later so those roles are  dynamic in Pgauto failover. You can also use some of the commands presented in the  slide to figure out the current properties of the system. Also included in Pgauto failover is  online membership changes just like I said before when the node fails then something happens and  then the membership changes from primary to secondary or secondary to primary things like that.  We also include support for maintenance operation. A node that is ongoing maintenance cannot be a  candidate for failover of course. So you can enable maintenance on a node do a kernel security  upgrade for example and then when it's back online you can disable maintenance for this node to be  a candidate for failover again when it's back to being ready for that. Included also in Pgauto  failover is network split detection and protection thanks to the monitor being there in the architecture  and thanks also to some smarts included in Pgauto CTL client itself. Pgauto CTL is going to manage  the post-grasqual instance and also check out some network properties and make some decisions about  the situation. You can even perform a promotion and target manually the node that you want to be  the new primary which might be useful if you're migrating from some data center to another or  if you change your current primary availability zone that's possible to do with a simple perform  promotion command in Pgauto failover. Not included yet in Pgauto failover is archiving and disaster  recovery setup. The idea is that in most of the existing archiving and disaster recovery solutions  the idea that roles are dynamic is not implemented. There is a walkthrough in the tutorial or in the  documentation to install the solution with a known current primary. It's hard to figure out what you  should do when this primary is going to change and then to make it automatic. So that's why I think  I believe that we should have something about that in Pgauto failover itself. And again configuration  management, extension management etc. It's not today part of what Pgauto failover is going to do  on your clusters in prediction. So that's it. That's the talk. That's my presentation today  post-grasqual architectures in prediction. Feel free to ask me questions now and have a good day. Thank you.
postgresql_postgres_architectures_in_production_0_179.png postgresql_postgres_architectures_in_production_180_539.png postgresql_postgres_architectures_in_production_540_971.png postgresql_postgres_architectures_in_production_972_1199.png postgresql_postgres_architectures_in_production_1200_1631.png postgresql_postgres_architectures_in_production_1632_1907.png postgresql_postgres_architectures_in_production_1908_2135.png postgresql_postgres_architectures_in_production_2136_3575.png postgresql_postgres_architectures_in_production_3576_4103.png postgresql_postgres_architectures_in_production_4104_4943.png postgresql_postgres_architectures_in_production_4944_5543.png postgresql_postgres_architectures_in_production_5544_7343.png postgresql_postgres_architectures_in_production_7344_8975.png postgresql_postgres_architectures_in_production_8976_9311.png postgresql_postgres_architectures_in_production_9312_14435.png postgresql_postgres_architectures_in_production_14436_17399.png postgresql_postgres_architectures_in_production_17400_21311.png postgresql_postgres_architectures_in_production_21312_21635.png postgresql_postgres_architectures_in_production_21636_21791.png postgresql_postgres_architectures_in_production_21792_23951.png postgresql_postgres_architectures_in_production_23952_25595.png postgresql_postgres_architectures_in_production_25596_26195.png postgresql_postgres_architectures_in_production_26196_26363.png postgresql_postgres_architectures_in_production_26364_26471.png postgresql_postgres_architectures_in_production_26472_27251.png postgresql_postgres_architectures_in_production_27252_28199.png postgresql_postgres_architectures_in_production_28200_29291.png postgresql_postgres_architectures_in_production_29292_30023.png postgresql_postgres_architectures_in_production_30024_31079.png postgresql_postgres_architectures_in_production_31080_32231.png postgresql_postgres_architectures_in_production_32232_33563.png postgresql_postgres_architectures_in_production_33564_33755.png postgresql_postgres_architectures_in_production_33756_33875.png postgresql_postgres_architectures_in_production_33876_34267.png