Beyond The Hype: One Year Running Addresscloud on 100% Serverless
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Holderness, Tomas
Serverless enables geospatial developers to build applications without worrying about servers or containers. In this session we will look at the advantages and challenges of serverless for geospatial, drawing on Addresscloud's experience as an early adopter and insights gained from using serverless to power production geocoding and location intelligence services. What’s it like to run a geospatial service without any servers? Addresscloud is a Software-as-a-Service for geographic risk and location intelligence. Addresscloud is powered by FOSS; using a combination of PostGIS, COGs, Elasticsearch, Vector Tiles, MapLibre GL and GeoJSON our APIs are used by millions of consumers in the insurance, finance and logistics sectors across Europe and North America. In 2020 we completed a re-architecture of our service to become 100% serverless. As early adopters of serverless for geospatial this talk will explore the advantages of serverless, demonstrating how it has improved our scalability, reliability and consistency of service, and enabled us to become more competitive. We will also share our experience of the transition and the challenges faced, particularly around developer learning curves, system observability and complexity. The presentation will be useful for members of the community looking to use their favourite FOSS tools to build geospatial applications in the cloud.
 Your next speaker is Thomas. Hello Thomas, who are you? Hello, I'm good thanks. Yeah, Thomas, you talk about Serbless and he is the CTO of address cloud. Where he leads research and development of geographic risks and location intelligence services. Yeah, and Thomas, your turn. Brilliant, thanks. Just going to share my screen. How's that? Looks good. Great, so thanks everyone for joining my talk. And as has been said, my name is Thomas Holderness and I work at address cloud. And today I wanted to talk to you about our experience at address cloud of running our software as a service business on 100% Serbless architectures. So a number of phospho G events now, phospho G UK online last year, I think, phospho G at Bucharest. I've zoomed in to our architecture and talked about very specific configuration and use cases of free and open source tools like post GIS and data formats like cloud optimized geotifs and how we're using them to serve large amounts of geographic data in response to customer queries. And the focus of this presentation is to kind of take a step back or zoom out a little bit and think about actually how what our experience been like and kind of share our experience and kind of advocate a little bit for some of the things that we've done and also kind of give a heads up on some of the challenges that we've had. Because I think the space is only going to see more investment in the future going forwards. So there's six things that I'd like to talk to you about today that are kind of facets of our serverless experience. The first is cost. I was involved in a Twitter question, a Twitter discussion a couple of months back and someone was like, yeah, but what is the cost difference between running virtual machines or running a suite of Docker containers or running EC2s or even physical machines versus a service architecture. And that's quite a hard thing to measure and to quantify. But I've got some examples that I'm going to work through because I think that for any business that's running a service, the cost of cloud infrastructure is key. And then we move on to scalability. And this is hand in hand with cost really because you need to understand the capacity of your system to know what throughput your customers can get from your service. And you need to be able to deal with that maximum throughput. And related to that is latency. You need to know that as your customer demands increase upon your service that you're going to be able to respond to those queries without the latency increasing. And at the bottom here, we've got service. And by service, I mean what service can we offer our customers and what is the quality of that service. And it's the service that I'm offering actually benefiting because of this serverless architecture. Going to look at infrastructure as code, which is one of the challenges and was great to see in the previous presentation, some examples of a Terraform plugin to deploy a Geo server. That's really cool because we use Terraform as well. And I'm going to talk a bit about why we chose that and how that was really a key kind of learning experience for us about adopting this kind of serverless architecture. And then lastly, observability. So if I haven't got a physical machine to log into, it's really important that I can adapt to that. And then to it's really important that I can observe that my system is responding in the way that I think that it should be, that the latency isn't increasing, that the scaling is happening, and that the costs aren't spiraling out of control. So that observability piece is really key. And I'm going to show just our flow line of how we do that, which might be helpful to some of you. But it would be remiss of me to not talk about Geospatial, Geospatial Conference, and maybe even show a map or a screenshot of a map. And so for those of you that haven't heard of us, I just wanted to really give a brief overview about Address Cloud, because it really sets in context a lot of the things that I'm going to talk to you about. So the first thing you should know is that we're a software as a service company. We're small and there's five of us. We're based in the United Kingdom. We're 100% employee owned. And we work in the insurance sector in financial services with banks, in logistics, and also in property survey. Those are kind of our key markets. We do about 10 million transactions a month. We have about 400 users of our system. And we power some well-known brand names here in the UK. So if people are getting insurance quote, then lots of that processing is coming to us. And the reason that that is the case is because we really provide two key services. We provide geocoding, so entering an address and getting a location back on the earth's surface. And we provide property intelligence and a geographic risk assessment of that property. I'll show you some examples of that in a couple of slides. So it's really important to our customers. We've got quite strict service double agreements with them that our transactions are being processed really quickly. And that it's available all the time, because the public are essentially using our customer systems and that's backing onto us. And if we can't resolve that piece of information, that geographic query fast enough, then that's potentially a lost customer or a lost sale. So it's really important that our services on and available at that capacity all the time. So it's kind of an infographic of a little bit of what we do. And we've got an address up here at the top. And so we've taken that address in, and we've geocoded it so we know where it is on the earth's surface. And once we know where it is on the earth's surface, then we can try to understand what sort of property is at that address. Is it residential? Is it commercial? Is it mixed use? Is it government property? And is it a tower block? Or is it a family home? And once we understand what sort of property it is, we can move on to understand what perils might affect that property. That an insurer or someone in financial services might be interested in. Is that property at risk of flooding or of fire or wildfire in North America? Or of subsidence or earthquake? And then another kind of interesting geographical dimension that we're starting to add to the service is for insurers who are taking on that risk. How are they exposed in the neighborhood if they start to take on lots of risks nearby? Then if there was a flooding event, how would that impact them? So that's kind of what we do puts in context why we need to architect in the way that we do. Excuse me, there's a screenshot of one of our applications that shows a map. This is for an insurance underwriter that would come into a dress cloud to view a risk. And we've geocoded that risk and we can see that we've added some layers to that map. In this case is a flood risk of some nearby properties. This is actually a train station in the city near where I live. So we've geocoded that. We've got the point on the earth surface. We know what the property is. We've got some risk scores over here. And so all of the processes that have happened behind this application, including loading the vector tiles and the raster tiles that power these overlays, those are all coming through a dress clouds API. And we're now using map Libre to power this React application, which is great. So I didn't want to show an architecture diagram, but I did want to kind of show how all of this fits together because it gives a lot of context to our experience. And really it's important to note that we didn't lift and shift our existing solution into a serverless architecture. We've re-architected for a number of reasons really into a serverless architecture. So we've changed the way that our service operates. So on this slide, you'll see on the right hand side that we've got kind of four layers, and I'm going to start down here at the data layer. So we've got three principal sources of a data store that we work with. I've talked to Fossil for Gene Bucharest, and you can see the video and the slides are blog.addresscloud.com. I will share the link at the end about using cloud optimized geotifs. This is kind of a scalable data store when they're putting an Amazon S3 bucket as a way to query large, really large, complicated, continuous surface models and get handfuls of pixels back at any point in time. I also talked to Fossil for Gene Edinburgh about how we can use Postgres to and Post GIS within the Amazon Aurora service to have a serverless and scalable Post GIS suite. We use Elasticsearch, which is hosted by Elastic Clouds. It's not necessarily 100% serverless as you say, but it is a managed service for someone else that's doing the infrastructure management for us. My colleague Mark Vali has given a number of presentations about the nuts and bolts of using Elasticsearch as a geospatial data engine. So sitting on top of these three kind of scalable data stores, we've got some application logic. This was originally our monolithic JavaScript application in Node.js. It's now split into a series of Lambda functions. One of the things that we've done is think about, okay, so we've got these supposedly scalable spatial data stores in the back end, but we know that there are still some bottlenecks here. We know that to scale Aurora takes a couple of seconds. So that could be a potential bottleneck, for example. We know that the capacity of our Elastic Cloud can scale, but again, there's some latency associated with that. So one of the things that we've done as we've looked at the information that we've got and the service that we provide is we've actually pre-indexed lots of our data using the H3 geospatial index. And then we've put that data inside a DynamoDB table, which is a serverless database offering from Amazon Web Services. And it's a key value lookup. So very quickly, we can look up for any known location, any known property in the country, and we can look up its properties. And this is kind of like an all-you-can-eat database with as much capacity and speed as you could ever imagine. So that's great. So it's got some logic that decides which of these to query or potentially queries, all of them, and hopefully tries the cache first. And if not, falls back on these. We've got an API which manages our API keys and authentication. And then we've got a happy user sitting at the top here that's interacting with that API or is interacting with the desktop application or the web application I showed before. So that's kind of it in a snapshot. That's the way that we've set things up. So let's dive in and look at some of the kind of experiences and advantages. So the first one, kind of as I promised, was to look at cost. It's really hard to compare sort of before and after costs with our serverless architecture at least, because we changed so much in the development. So what I've done is I've picked two examples of things where I've tried to understand what the differences in cost would be between a serverless and non-serverless setup. And we could probably pick holes in some of this, but it's just kind of a, I guess it's a working case study. But what I wanted to kind of illustrate here is on the right, we've got some graphs. And this is some queries that are coming into our vector database. And that database is post GIS. It's hosted in Aurora Serverless. And we can see that over the month of March this year, we've spiked about 500 post GIS queries in an hour. So we're starting to tax the database. Customers come in and probably put some quite complicated query requests through our API. Potentially, we haven't got those values in the cache or they're really complicated shapes. And so we've had to fall back on a true spatial query. And you can see what's immediately happened here is that on our behalf, without us even knowing about it, Amazon has scaled that database for us. It's spiked up from two compute units to eight compute units. And then after those queries have kind of calmed down, and there's probably some data in the cache as well, those compute units have dropped off and we've gone back down to two. So what I've done here on the left is, so we spiked up to eight ACU. Obviously, we weren't using eight ACU over the entire two of the month. We were only using them probably for a couple of hours in that day, whilst we saw this peak load. So our bill for Aurora serverless that month was $114. Now, if we were to provision capacity in a traditional manner without any auto scaling, I estimate that we'd need a T3X large database type, so virtual machine from Amazon that's running Postgres. Relational database service is a managed service that you can use from Amazon, but you still have to control the capacity and the scaling yourself. So you still have to do some work there. And I've also selected an instance which allows cross availability zone replication, which is something that you get out of the box with serverless. So if one of these, there's actually three database instances in this case, and if one of them goes down, the master goes down, then it'll fall back on one of the other two. And the same happens with Aurora, and I've actually got an example of that happening in a few slides time. But you can see the price difference, there is a price difference there, and you could think about there are some ways you could reduce this price probably. But the point being that because you're trying to be at capacity all the time to make sure that your customers, if they come knocking, can get that performance, you've got a four times increase in cost if you were to run those databases all the time. Now, you could put a load of work in there to put auto scaling in to manage that, to think about different database operations or approaches. But the point is that you can get that out of the box here and save money without really having to do anything. And it's just post-GIS, under the hood or it's post-GIS compatible, at least. So that's one example. The second example is comparing our compute capacity. So what I did is I added up all of our function invocations for every Lambda function that we use. So this is all of our environments development production, it's helping our customers do their integration. So it's our sandbox environment, it's all of our back office processes now, which we also use Lambda functions for, pretty much gone all in on this. The only thing that isn't captured by this sort of processing is our data, our data preprocessing, which tends to have very long jobs that run over a couple of days. And for those, we still use a combination of EC2 instances and Docker instances. And those aren't captured here. But for all of those, so we've done 20 million function invocations and Amazon has kind of come up with this magic number of said that in that one month, we had 82 days worth of computing time. So we did a lot of computing in that month over the month of March. And so we've got these 20 million transactions, it cost us 56 bucks, which is kind of incredible. So that's all about everything that's customer facing, everything that's in development and every back office process that happens in Amazon Web Services for 60 bucks. If we were to replicate that across the three services and the back office services, I think we have this as kind of an estimate that I came up with today, we'd need about 90 to medium EC2s to meet capacity to meet peak capacity, you can see these peaks here. And so that would cost us a couple of hundred bucks a month, we could probably reduce that if we went for reserved instances, and we had a conversation with Amazon sales guys, and you know, we could probably hack all that down. But the point being that because we're not paying for this stuff to be on all the time, we're only paying for what we use, even though we're using a lot of it, still very cheap. So I would kind of advocate there that there are cost savings to be had, but it's got to be coupled with the way that you think about architecting your application. And that kind of leads nicely into scalability and latency. And I was a bit worried yesterday when I started writing my slides, because well, I'd left it to a little bit too late, and I normally try to be a bit more organized than that. And then I was also thinking, we haven't really thought about this service for or any of our services operationally, because we've been busy doing other things. So we've been onboarding customers, we've been improving speed, we've been adding new functionality. But the service itself has just been chugging away, it's just been serving up those customer requests. And I think that's kind of one of the real benefits, the hidden benefits of choosing a serverless environment. So the purple graph across the top shows our geocoding requests coming in, again, for the month of March this year, I've just picked it because March seemed to be a good month, the graph seemed to look nice. So that's what we're doing. We've got to about, looks like 43,000 requests over about an hour or so on one of the days, that's kind of the peak that month. So this is, that's what this activity is showing. So it drops off in the evenings, you know, when there's a few requests coming in from customers, and then it peaks again in the day when people are using the system to do their work. And then down here on the second graph, the red line shows the latency of that service. So we've got geocoding requests coming in across the top. And what's quite interesting is that it doesn't appear to be a relationship or any similarity between the changing in latency of the service and the number of demand that's in the top graph. I'd say it would also point out that this latency graph is a smooth average. So this is by no means perfect. And the tool that we use here, I could have dived in more and seeing if we could get a distribution plotted on this graph, but I didn't quite have time before this. But you can see that, you know, our peak latency is still an average of 240 milliseconds. And our fastest response time was about, on average, about 80 milliseconds over the month. So it's pretty good, we're well within our SLA for our customers. And we're able to gobble up these increased transaction loads of bits, these big jobs that customers are checking at us. And we've got no sweat about the latency. So service, what does that mean? That means that I kind of have this metric of how many times do I have to get out of bed to deal with the to adjust cards operational side of things. And I'm pleased to report that this only happened once in the last 365 days. So I counted up all of our external API transactions and all of our internal services that have also got APIs. And we did 142 million API transactions over the year, across every environment, across all of our APIs in the stack, so both dev and production. And overall of those transactions, we had one production alarm. Now, frustratingly, this actually happened whilst I was on holiday for the one week that I was on holiday over the summer, actually away from my house. But it was fine. I still had some internet and there's also a team of us, so we can also share out when these events happen. So we've only had one event. And what happened was our Postgres database experienced intermittent connectivity for an hour. And that was because Amazon had made some configuration errors or had some configuration problems in a couple of their data centers. So what they did without us even having to do anything is that that master node was in a data center that was having problems, they shifted that master node to another data center, still in the same region, but up to 100 miles away, they shifted that across and then they started to split the traffic over. And actually, because of the way our applications are protected, we didn't none of our customers even notice that there was an issue. This was just something that was picked up by our internal monitoring. And they had that fixed within the hour and within about 30 minutes, we actually saw that the error rate decreasing. So kind of an example of all of that stuff going on throughout the year. And we're all busy people and we're all kind of building our services or running our projects or trying to deliver the tools and contribute to FOS4G. Really nice to be able to know that kind of in safe hands of the cloud provider for most of the work and that really they're taking the lead and responsibility on maintaining that uptime. So lastly, sort of the five of the six things to mention infrastructure is code. This was a steep learning curve for us and something that we started a couple of years ago, so a long time before we went fully serverless. We chose to use Terraform for a variety of reasons, but predominantly because it's very robust, it's declarative, which I really like. And it also supports more than just AWS, so you can use it to configure lots of different types of infrastructure, which is great in lots of different clouds. And so we actually ended up investing a lot of time in this, building our own deployment pipeline so that we understood what went on, making sure that we could version control all of this infrastructure, which we've defined inside Amazon Web Services, so that at any point in time we can see exactly who has changed what and what our infrastructure is represented. So we've got lots of code files that represent all of the suite of tools that we use and how they're all connected together. And we can ship that as part of our CI CD process. So a developer can come in, make a commit either to a logic change in the application or to some infrastructure, push that to GitHub, have some tests run, if everything passes, that gets pushed to Terraform. If everything passes there, that actually gets deployed against that infrastructure and we've got a process of making sure that we're happy with that in the development stage before that gets pushed to production. And I've written about and talked about this in the past, you can grab that blog from our website. And then lastly, observability. In the last couple of minutes of this presentation, observability is really important when you're thinking about a service application. And I think it shouldn't be thought about as an afterthought, it needs to be thought about as part of your ongoing IT or business processes. Because you don't have a normal service coming from our traditional monolithic application where we're backing onto PostGIS and Elasticsearch and we just had this thing that was running queries, we could dive into that box or those boxes and we could grab the logs and see how those queries were running and what queries are being executed. Obviously, with lots of the tools that we work with, there is no physical box or virtual box that we can log into. So we need to capture those logs if there are any, we need to capture the metrics around those transactions. Ideally, do some of that end to end. It's not something that we do well, but something that I think we could definitely improve upon. So we want to capture all of that information and then we want to be able to interrogate it so that we can actually query to understand what was going on and we can produce some of the graphs that you've seen in this presentation. So there are a few different ways of doing that. And this is the way that we chose to do it. We used the Postman library. They have a core module, a library called Newman, which is Apache licensed, an open source bit of kit. We took that, we wrote some code around it to run a test suite against our API that's running every 60 seconds, every 30 seconds. And that's running all of these tests and we're pumping those logs into an Amazon tool called CloudWatch so we can see how those tests are performing. We can grab those metrics and those logs into Grafana, which is another open source bit of kit, and that allows us to see in real time what's going on. And if any of those graphs are over a certain threshold for a given period, then Grafana sends us a ping via a pager duty, which is what wakes you up when you're on holiday. So that's kind of how we've architected our observability. As I say, lots of different ways to do this, but just wanted to make the point that it's a key thing about thinking about and it's not necessarily an easy challenge to solve. So in summary, we're a big advocate for serverless because we've managed to re-architect our SaaS application to serve the needs of our customers and to serve the needs of our business and for us to be able to continue to scale and to operate at scale and have low latencies and basically make sure that we're doing what we need to do to make our customers happy, which is kind of the key goal. And really, I'm happy because we've managed to take some of the best bits of FOS that we've been used to using, be it GeoJSON, tiles, COGS, Factor Tiles are in there as well. I haven't talked about those today, but maybe another presentation. We've combined all of that kind of best bits that we've managed to get it together in a nearly pure play serverless architecture in AWS and hopefully we'll continue to grow and we'll be at FOS 4G next year as well. So that's me. If you've got any questions, then pop them in the chat or feel free to send me an email. Thanks very much. Thanks Thomas. I have one question. It's a personal question. Yeah, it's I imagine you collect data from a very internal external source. How is the ETL process to keep everything up to date? Is everything serverless too? Well, that is a great question. And traditionally, no, we had a series of scripts and we were running those on an EC2 box. What we're actually just building at the moment this month, Mike has joined us and he's built some tech using AWS Batch, which is a dockerized batch processing tool where you can have a queue of tasks and then you can have a fleet of instances that run docker containers. And so he's doing some brilliant work at the moment to basically operationalize that process so that exactly that so that all of those ETL processes don't have to be run manually or semi-automated in a semi-automated manner as they are now, but can actually we can just have a queue of tasks that can be triggered by different events or by different time periods and they can go grab the data and they can then wait for the availability of the processing pool and then they can crack on and then they can be pushing the data artifacts out into the testing environment and then eventually to production. So that's something that we're working on actually at the moment, but yeah, it's an interesting problem to have. Okay, thank you. Another question is, did you find AWS lambda coding sometimes hard-going what with library size layers and etc? Yes, that is a great challenge. We do for the smaller functions that are in JavaScript, we just we use webpack to compile down that code to basically build a JavaScript executable if you like that then gets uploaded. For big geo libraries like the stuff that we do with raster actually give it it's on the blog but we talk to give a talk about it earlier this year about Hammerson's new Docker container lambda environment where you can use a Docker image instead of packaging up using sort of traditional packaging tools like instead of having a zip file you can create a Docker image and I've actually did a demo of using and we're actually using in production now the Docker image that builds rasterio so that we can use rasterio in python to do our query our cloud optimized geotifs and that's working really really well it's actually slightly faster than the traditional python zip file method operationally. Okay, you have time for another question. Well, do you think serverless architecture could work for latency sensitive applications like the the high vehicular routing that typically really keeping a ref in memory for quicker response? Yeah, I think I mean anything where you've got a status you've got like a stateful transaction where you're reliant on a server side process managing state on behalf of a user's process is going to be trickier with the with the sort of traditional API gateway and lambda architecture that I've shown. That being said I think you you know there's a number of options of things that you could do there about okay why do you need to keep that why do you need to keep that thing in memory and or is there an in-memory cache like memcache that you could use to store that to sort of back onto there. It's an interesting one that's something that I've never experimented with is API gateways socket sockets so as well as just doing like restful interactions you can do longer lived connections and I think that that would be really interesting to sort of play around with that and so say if you've got a continuous connection between machine and your service you could stream data between the two but what the back end would look like that I don't know I haven't talked about it. Okay, thank you a lot for your answer Thomas. Did you have anything more to do to talk? No that was brilliant thank you and thanks for everyone for joining. Okay thank you.
 Your next speaker is Thomas.  Hello Thomas, who are you?  Hello, I'm good thanks.  Yeah, Thomas, you talk about Serbless and he is the CTO of address cloud.  Where he leads research and development of geographic risks and location intelligence services.  Yeah, and Thomas, your turn.  Brilliant, thanks.  Just going to share my screen.  How's that?  Looks good.  Great, so thanks everyone for joining my talk.  And as has been said, my name is Thomas Holderness and I work at address cloud.  And today I wanted to talk to you about our experience at address cloud of  running our software as a service business on 100% Serbless architectures.  So a number of phospho G events now,  phospho G UK online last year, I think, phospho G at Bucharest.  I've zoomed in to our architecture and talked about very specific  configuration and use cases of free and open source tools like post GIS and data formats  like cloud optimized geotifs and how we're using them to serve large amounts of geographic data  in response to customer queries.  And the focus of this presentation is to kind of take a step back or zoom out a little bit  and think about actually how what our experience been like and kind of share  our experience and kind of advocate a little bit for some of the things that we've done and also  kind of give a heads up on some of the challenges that we've had.  Because I think the space is only going to see more investment in the future going forwards.  So there's six things that I'd like to talk to you about today that are kind of facets of our  serverless experience. The first is cost. I was involved in a Twitter question, a Twitter  discussion a couple of months back and someone was like, yeah, but what is the cost difference  between running virtual machines or running a suite of Docker containers or running EC2s  or even physical machines versus a service architecture. And that's quite a hard thing  to measure and to quantify. But I've got some examples that I'm going to work through because  I think that for any business that's running a service, the cost of cloud infrastructure  is key. And then we move on to scalability. And this is hand in hand with cost really because  you need to understand the capacity of your system to know what throughput your customers  can get from your service. And you need to be able to deal with that maximum throughput.  And related to that is latency. You need to know that as your customer demands increase  upon your service that you're going to be able to respond to those queries without the latency  increasing. And at the bottom here, we've got service. And by service, I mean what service  can we offer our customers and what is the quality of that service. And it's the service that I'm  offering actually benefiting because of this serverless architecture.  Going to look at infrastructure as code, which is one of the challenges and was great to see in the  previous presentation, some examples of a Terraform plugin to deploy a Geo server.  That's really cool because we use Terraform as well. And I'm going to talk a bit about why we  chose that and how that was really a key kind of learning experience for us about adopting  this kind of serverless architecture. And then lastly, observability. So if I haven't got a  physical machine to log into, it's really important that I can adapt to that. And then  to it's really important that I can observe that my system is responding in the way that I think  that it should be, that the latency isn't increasing, that the scaling is happening,  and that the costs aren't spiraling out of control. So that observability piece is really key. And I'm  going to show just our flow line of how we do that, which might be helpful to some of you.  But it would be remiss of me to not talk about Geospatial, Geospatial Conference,  and maybe even show a map or a screenshot of a map. And so for those of you that haven't heard  of us, I just wanted to really give a brief overview about Address Cloud, because it really  sets in context a lot of the things that I'm going to talk to you about. So the first thing  you should know is that we're a software as a service company. We're small and there's five of  us. We're based in the United Kingdom. We're 100% employee owned. And we work in the insurance sector  in financial services with banks, in logistics, and also in property survey. Those are kind of  our key markets. We do about 10 million transactions a month. We have about 400 users of our system.  And we power some well-known brand names here in the UK. So if people are getting insurance quote,  then lots of that processing is coming to us. And the reason that that is the case is because we  really provide two key services. We provide geocoding, so entering an address and getting a  location back on the earth's surface. And we provide property intelligence and a geographic  risk assessment of that property. I'll show you some examples of that in a couple of slides. So  it's really important to our customers. We've got quite strict service double agreements with them  that our transactions are being processed really quickly. And that it's available all the time,  because the public are essentially using our customer systems and that's backing onto us. And  if we can't resolve that piece of information, that geographic query fast enough, then that's  potentially a lost customer or a lost sale. So it's really important that our services on and  available at that capacity all the time. So it's kind of an infographic of a little bit of what  we do. And we've got an address up here at the top. And so we've taken that address in, and we've  geocoded it so we know where it is on the earth's surface. And once we know where it is on the earth's  surface, then we can try to understand what sort of property is at that address. Is it residential?  Is it commercial? Is it mixed use? Is it government property? And is it a tower block?  Or is it a family home? And once we understand what sort of property it is, we can move on to  understand what perils might affect that property. That an insurer or someone in financial services  might be interested in. Is that property at risk of flooding or of fire or wildfire in North America?  Or of subsidence or earthquake? And then another kind of interesting geographical dimension that  we're starting to add to the service is for insurers who are taking on that risk. How are  they exposed in the neighborhood if they start to take on lots of risks nearby? Then if there was  a flooding event, how would that impact them? So that's kind of what we do puts in context  why we need to architect in the way that we do. Excuse me, there's a screenshot of one of our  applications that shows a map. This is for an insurance underwriter that would come into a  dress cloud to view a risk. And we've geocoded that risk and we can see that we've added some layers  to that map. In this case is a flood risk of some nearby properties. This is actually a train  station in the city near where I live. So we've geocoded that. We've got the point on the earth  surface. We know what the property is. We've got some risk scores over here. And so all of the  processes that have happened behind this application, including loading the vector tiles and the  raster tiles that power these overlays, those are all coming through a dress clouds API. And we're  now using map Libre to power this React application, which is great. So I didn't want to show an  architecture diagram, but I did want to kind of show how all of this fits together because it gives  a lot of context to our experience. And really it's important to note that we didn't lift and  shift our existing solution into a serverless architecture. We've re-architected for a number  of reasons really into a serverless architecture. So we've changed the way that our service operates.  So on this slide, you'll see on the right hand side that we've got kind of four layers, and I'm  going to start down here at the data layer. So we've got three principal sources of a data store  that we work with. I've talked to Fossil for Gene Bucharest, and you can see the video and the slides  are blog.addresscloud.com. I will share the link at the end about using cloud optimized geotifs.  This is kind of a scalable data store when they're putting an Amazon S3 bucket as a way to query  large, really large, complicated, continuous surface models and get handfuls of pixels back at  any point in time. I also talked to Fossil for Gene Edinburgh about how we can use Postgres to  and Post GIS within the Amazon Aurora service to have a serverless and scalable Post GIS suite.  We use Elasticsearch, which is hosted by Elastic Clouds. It's not necessarily 100%  serverless as you say, but it is a managed service for someone else that's doing the infrastructure  management for us. My colleague Mark Vali has given a number of presentations about the nuts  and bolts of using Elasticsearch as a geospatial data engine. So sitting on top of these three  kind of scalable data stores, we've got some application logic. This was originally our  monolithic JavaScript application in Node.js. It's now split into a series of Lambda functions.  One of the things that we've done is think about, okay, so we've got these supposedly scalable  spatial data stores in the back end, but we know that there are still some bottlenecks here. We know  that to scale Aurora takes a couple of seconds. So that could be a potential bottleneck, for example.  We know that the capacity of our Elastic Cloud can scale, but again, there's some latency associated  with that. So one of the things that we've done as we've looked at the information that we've got  and the service that we provide is we've actually pre-indexed lots of our data using the H3 geospatial  index. And then we've put that data inside a DynamoDB table, which is a serverless database  offering from Amazon Web Services. And it's a key value lookup. So very quickly, we can look up for  any known location, any known property in the country, and we can look up its properties. And  this is kind of like an all-you-can-eat database with as much capacity and speed as you could ever  imagine. So that's great. So it's got some logic that decides which of these to query or potentially  queries, all of them, and hopefully tries the cache first. And if not, falls back on these.  We've got an API which manages our API keys and authentication. And then we've got a happy user  sitting at the top here that's interacting with that API or is interacting with the desktop application  or the web application I showed before. So that's kind of it in a snapshot. That's the way that  we've set things up. So let's dive in and look at some of the kind of experiences and advantages.  So the first one, kind of as I promised, was to look at cost. It's really hard to compare  sort of before and after costs with our serverless architecture at least, because we changed so  much in the development. So what I've done is I've picked two examples of things where I've  tried to understand what the differences in cost would be between a serverless and non-serverless  setup. And we could probably pick holes in some of this, but it's just kind of a, I guess it's a  working case study. But what I wanted to kind of illustrate here is on the right, we've got some  graphs. And this is some queries that are coming into our vector database. And that database is  post GIS. It's hosted in Aurora Serverless. And we can see that over the month of March this year,  we've spiked about 500 post GIS queries in an hour. So we're starting to tax the database.  Customers come in and probably put some quite complicated query requests through our API.  Potentially, we haven't got those values in the cache or they're really complicated shapes. And  so we've had to fall back on a true spatial query. And you can see what's immediately happened here  is that on our behalf, without us even knowing about it, Amazon has scaled that database for us.  It's spiked up from two compute units to eight compute units. And then after those queries have  kind of calmed down, and there's probably some data in the cache as well, those compute units  have dropped off and we've gone back down to two. So what I've done here on the left is, so we spiked  up to eight ACU. Obviously, we weren't using eight ACU over the entire two of the month. We were  only using them probably for a couple of hours in that day, whilst we saw this peak load. So our  bill for Aurora serverless that month was $114. Now, if we were to provision capacity in a traditional  manner without any auto scaling, I estimate that we'd need a T3X large database type, so virtual  machine from Amazon that's running Postgres. Relational database service is a managed service  that you can use from Amazon, but you still have to control the capacity and the scaling yourself.  So you still have to do some work there. And I've also selected an instance which allows cross  availability zone replication, which is something that you get out of the box with serverless. So  if one of these, there's actually three database instances in this case, and if one of them goes  down, the master goes down, then it'll fall back on one of the other two. And the same happens with  Aurora, and I've actually got an example of that happening in a few slides time. But you can see  the price difference, there is a price difference there, and you could think about there are some  ways you could reduce this price probably. But the point being that because you're trying to be at  capacity all the time to make sure that your customers, if they come knocking, can get that  performance, you've got a four times increase in cost if you were to run those databases all the  time. Now, you could put a load of work in there to put auto scaling in to manage that, to think  about different database operations or approaches. But the point is that you can get that out of the  box here and save money without really having to do anything. And it's just post-GIS, under the  hood or it's post-GIS compatible, at least. So that's one example. The second example is  comparing our compute capacity. So what I did is I added up all of our function invocations for  every Lambda function that we use. So this is all of our environments development production,  it's helping our customers do their integration. So it's our sandbox environment, it's all of our  back office processes now, which we also use Lambda functions for, pretty much gone all in on this.  The only thing that isn't captured by this sort of processing is our data, our data preprocessing,  which tends to have very long jobs that run over a couple of days. And for those, we still  use a combination of EC2 instances and Docker instances. And those aren't captured here.  But for all of those, so we've done 20 million function invocations and Amazon has kind of come  up with this magic number of said that in that one month, we had 82 days worth of computing time.  So we did a lot of computing in that month over the month of March. And so we've got these 20  million transactions, it cost us 56 bucks, which is kind of incredible. So that's all about everything  that's customer facing, everything that's in development and every back office process that  happens in Amazon Web Services for 60 bucks. If we were to replicate that across the three  services and the back office services, I think we have this as kind of an estimate that I came  up with today, we'd need about 90 to medium EC2s to meet capacity to meet peak capacity,  you can see these peaks here. And so that would cost us a couple of hundred bucks a month, we could  probably reduce that if we went for reserved instances, and we had a conversation with Amazon  sales guys, and you know, we could probably hack all that down. But the point being that because  we're not paying for this stuff to be on all the time, we're only paying for what we use, even  though we're using a lot of it, still very cheap. So I would kind of advocate there that there are  cost savings to be had, but it's got to be coupled with the way that you think about  architecting your application. And that kind of leads nicely into scalability and latency.  And I was a bit worried yesterday when I started writing my slides, because well, I'd left it to  a little bit too late, and I normally try to be a bit more organized than that. And then I was also  thinking, we haven't really thought about this service for or any of our services operationally,  because we've been busy doing other things. So we've been onboarding customers, we've been  improving speed, we've been adding new functionality. But the service itself has just been chugging  away, it's just been serving up those customer requests. And I think that's kind of one of the  real benefits, the hidden benefits of choosing a serverless environment. So the purple graph  across the top shows our geocoding requests coming in, again, for the month of March this year,  I've just picked it because March seemed to be a good month, the graph seemed to look nice.  So that's what we're doing. We've got to about, looks like 43,000 requests over about an hour or so  on one of the days, that's kind of the peak that month. So this is, that's what this activity is  showing. So it drops off in the evenings, you know, when there's a few requests coming in from  customers, and then it peaks again in the day when people are using the system to do their work.  And then down here on the second graph, the red line shows the latency of that service. So we've  got geocoding requests coming in across the top. And what's quite interesting is that it doesn't  appear to be a relationship or any similarity between the changing in latency of the service  and the number of demand that's in the top graph. I'd say it would also point out that this latency  graph is a smooth average. So this is by no means perfect. And the tool that we use here,  I could have dived in more and seeing if we could get a distribution plotted on this graph, but  I didn't quite have time before this. But you can see that, you know, our peak latency is still  an average of 240 milliseconds. And our fastest response time was about, on average, about 80  milliseconds over the month. So it's pretty good, we're well within our SLA for our customers.  And we're able to gobble up these increased transaction loads of bits, these big jobs that  customers are checking at us. And we've got no sweat about the latency. So service, what does  that mean? That means that I kind of have this metric of how many times do I have to get out  of bed to deal with the to adjust cards operational side of things. And I'm pleased to report that  this only happened once in the last 365 days. So I counted up all of our external API transactions  and all of our internal services that have also got APIs. And we did 142 million API transactions  over the year, across every environment, across all of our APIs in the stack, so both dev and  production. And overall of those transactions, we had one production alarm. Now, frustratingly,  this actually happened whilst I was on holiday for the one week that I was on holiday over the  summer, actually away from my house. But it was fine. I still had some internet and there's also  a team of us, so we can also share out when these events happen. So we've only had one event. And  what happened was our Postgres database experienced intermittent connectivity for an hour. And that  was because Amazon had made some configuration errors or had some configuration problems in a  couple of their data centers. So what they did without us even having to do anything is that  that master node was in a data center that was having problems, they shifted that master node  to another data center, still in the same region, but up to 100 miles away, they shifted that across  and then they started to split the traffic over. And actually, because of the way our applications  are protected, we didn't none of our customers even notice that there was an issue. This was just  something that was picked up by our internal monitoring. And they had that fixed within the  hour and within about 30 minutes, we actually saw that the error rate decreasing. So kind of an  example of all of that stuff going on throughout the year. And we're all busy people and we're all  kind of building our services or running our projects or trying to deliver the tools and  contribute to FOS4G. Really nice to be able to know that kind of in safe hands of the cloud  provider for most of the work and that really they're taking the lead and responsibility on  maintaining that uptime. So lastly, sort of the five of the six things to mention infrastructure  is code. This was a steep learning curve for us and something that we started a couple of years  ago, so a long time before we went fully serverless. We chose to use Terraform for a variety of reasons,  but predominantly because it's very robust, it's declarative, which I really like.  And it also supports more than just AWS, so you can use it to configure lots of different types of  infrastructure, which is great in lots of different clouds. And so we actually ended up investing  a lot of time in this, building our own deployment pipeline so that we understood what went on,  making sure that we could version control all of this infrastructure, which we've defined inside  Amazon Web Services, so that at any point in time we can see exactly who has changed what and what  our infrastructure is represented. So we've got lots of code files that represent all of the suite  of tools that we use and how they're all connected together. And we can ship that as part of our  CI CD process. So a developer can come in, make a commit either to a logic change in the application  or to some infrastructure, push that to GitHub, have some tests run, if everything passes,  that gets pushed to Terraform. If everything passes there, that actually gets deployed against  that infrastructure and we've got a process of making sure that we're happy with that in the  development stage before that gets pushed to production. And I've written about and talked  about this in the past, you can grab that blog from our website. And then lastly, observability.  In the last couple of minutes of this presentation, observability is really important when you're  thinking about a service application. And I think it shouldn't be thought about as an afterthought,  it needs to be thought about as part of your ongoing IT or business processes. Because you  don't have a normal service coming from our traditional monolithic application where we're  backing onto PostGIS and Elasticsearch and we just had this thing that was running queries,  we could dive into that box or those boxes and we could grab the logs and see how those queries were  running and what queries are being executed. Obviously, with lots of the tools that we work  with, there is no physical box or virtual box that we can log into. So we need to capture those logs  if there are any, we need to capture the metrics around those transactions. Ideally, do some of  that end to end. It's not something that we do well, but something that I think we could  definitely improve upon. So we want to capture all of that information and then we want to be able  to interrogate it so that we can actually query to understand what was going on and we can produce  some of the graphs that you've seen in this presentation. So there are a few different  ways of doing that. And this is the way that we chose to do it. We used the Postman library.  They have a core module, a library called Newman, which is Apache licensed, an open source bit of  kit. We took that, we wrote some code around it to run a test suite against our API that's running  every 60 seconds, every 30 seconds. And that's running all of these tests and we're pumping  those logs into an Amazon tool called CloudWatch so we can see how those tests are performing.  We can grab those metrics and those logs into Grafana, which is another open source bit of kit,  and that allows us to see in real time what's going on. And if any of those graphs are over a  certain threshold for a given period, then Grafana sends us a ping via a pager duty, which is what  wakes you up when you're on holiday. So that's kind of how we've architected our observability.  As I say, lots of different ways to do this, but just wanted to make the point that it's a key thing  about thinking about and it's not necessarily an easy challenge to solve. So in summary, we're a  big advocate for serverless because we've managed to re-architect our SaaS application to serve the  needs of our customers and to serve the needs of our business and for us to be able to continue  to scale and to operate at scale and have low latencies and basically make sure that we're  doing what we need to do to make our customers happy, which is kind of the key goal. And really,  I'm happy because we've managed to take some of the best bits of FOS that we've been used to using,  be it GeoJSON, tiles, COGS, Factor Tiles are in there as well. I haven't talked about those today,  but maybe another presentation. We've combined all of that kind of best bits that we've managed to  get it together in a nearly pure play serverless architecture in AWS and hopefully we'll continue  to grow and we'll be at FOS 4G next year as well. So that's me. If you've got any questions, then  pop them in the chat or feel free to send me an email. Thanks very much.  Thanks Thomas. I have one question. It's a personal question. Yeah, it's  I imagine you collect data from a very internal external source. How is the ETL process to keep  everything up to date? Is everything serverless too? Well, that is a great question. And  traditionally, no, we had a series of scripts and we were running those on an EC2 box.  What we're actually just building at the moment this month, Mike has joined us and he's built  some tech using AWS Batch, which is a dockerized batch processing tool where you can have a queue  of tasks and then you can have a fleet of instances that run docker containers. And so he's doing some  brilliant work at the moment to basically operationalize that process so that exactly that so that all  of those ETL processes don't have to be run manually or semi-automated in a semi-automated  manner as they are now, but can actually we can just have a queue of tasks that can be triggered  by different events or by different time periods and they can go grab the data and they can then  wait for the availability of the processing pool and then they can crack on and then they can be  pushing the data artifacts out into the testing environment and then eventually to production.  So that's something that we're working on actually at the moment, but yeah, it's an interesting  problem to have. Okay, thank you. Another question is, did you find AWS lambda coding sometimes  hard-going what with library size layers and etc? Yes, that is a great challenge. We do for  the smaller functions that are in JavaScript, we just we use webpack to compile down that code to  basically build a JavaScript executable if you like that then gets uploaded. For big geo libraries  like the stuff that we do with raster actually give it it's on the blog but we talk to give a talk  about it earlier this year about Hammerson's new Docker container lambda environment where you  can use a Docker image instead of packaging up using sort of traditional packaging tools like  instead of having a zip file you can create a Docker image and I've actually did a demo of using  and we're actually using in production now the Docker image that builds rasterio so that we can  use rasterio in python to do our query our cloud optimized geotifs and that's working really really  well it's actually slightly faster than the traditional python zip file method operationally.  Okay, you have time for another question. Well, do you think serverless architecture  could work for latency sensitive applications like the the high vehicular routing that typically  really keeping a ref in memory for quicker response?  Yeah, I think I mean anything where you've got a status you've got like a stateful transaction  where you're reliant on a server side process managing state on behalf of a user's process  is going to be trickier with the with the sort of traditional API gateway and lambda architecture  that I've shown. That being said I think you you know there's a number of options of things  that you could do there about okay why do you need to keep that why do you need to keep that thing  in memory and or is there an in-memory cache like memcache that you could use to store that  to sort of back onto there. It's an interesting one that's something that I've never experimented  with is API gateways socket sockets so as well as just doing like restful interactions you can do  longer lived connections and I think that that would be really interesting  to sort of play around with that and so say if you've got a continuous connection between machine  and your service you could stream data between the two but what the back end would look like that  I don't know I haven't talked about it. Okay, thank you a lot for your answer Thomas.  Did you have anything more to do to talk? No that was brilliant thank you and thanks  for everyone for joining. Okay thank you.
FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_0_131.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_132_191.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_192_239.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_240_287.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_288_371.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_372_455.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_456_503.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_504_551.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_552_1259.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_1260_1667.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_1668_3683.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_3684_6791.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_6792_7067.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_7068_7295.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_7296_7343.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_7344_7475.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_7476_8675.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_8676_8807.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_8808_8939.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_8940_9215.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_9216_10799.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_10800_11951.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_11952_16739.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_16740_17039.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_17040_21323.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_21324_24059.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_24060_24743.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_24744_27059.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_27060_27395.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_27396_30179.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_30180_30971.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_30972_32387.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_32388_33599.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_33600_35207.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_35208_36371.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_36372_36611.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_36612_36923.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_36924_39095.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_39096_41075.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_41076_43583.png FOSS4G_-_Beyond_The_Hype__One_Year_Running_Address_bwrKcaep0XQ_43584_43706.png