A 30x Boost In Speed: The First Weekend on Google Cloud
September 24, 2018 | 5 min read
We've just concluded our first weekend after migrating our servers from Amazon Web Services to Google Cloud. And as the person in charge of our server infrastructure, I couldn't be happier with the results. SpringboardVR's goal in hard numbers was to make 90% of operations execute in under 350ms, and I'm absolutely thrilled to announce we've met that goal.
Comparing peak traffic last week to this week, we've seen on average a speed boost of about 10x, and on some operations we've even seen a 30x increase in speed. This means you will notice a lot less 'weird' things happening, and in general your experience with SpringboardVR will feel a lot more smooth and snappy.
We reduced our overall latency by ~200ms with the move.
Let's dive deeper into this achievement and answer the following questions: 'Why was it slow in the first place?', 'what did SpringboardVR do to make it faster?', and finally 'what's next?'.
Why Was it Slow in the First Place?
Because SpringboardVR Has Grown Very Quickly
Over the last year we've nearly tripled the minutes being tracked on SpringboardVR. From about 8 million minutes to over 22 million.
That's a lot more minutes being tracked so far in 2018.
Another important metric to consider is that our traffic varies wildly based on the day of week. For example, Saturday is close to being twice as busy as the next busiest day, Sunday.
Minutes tracked in the last 30 days, grouped by day of week.
So, you can see that we had to combat wild swings in traffic during the week as well as huge growth. This was on top of keeping costs low and delivering the features that makes SpringboardVR unique. Our server infrastructure set up was meant for SpringboardVR circa 2017 and we quickly fell into 'catch up mode' when it came to issues of slowness. We knew we had to address the underlying causes as soon as possible…
What Did SpringboardVR Do to Make it Faster?
Better Visibility, Faster Code, and More Power
People who are familiar with Cloud Providers like Amazon Web Services (AWS) and Google Cloud are likely thinking 'why are you making such a big deal about this move?'. They would be right to be skeptical about the migration, because Google Cloud is not inherently faster or cheaper than AWS. The reason we are making such a big deal about this is because we took everything we learned from running the servers on AWS and made our setup on Google Cloud better in every way for SpringboardVR.
It starts with visibility. On AWS we didn't know much about our servers other than the basics like CPU and Memory. Even worse, our platform logs would retain only the last hour of information, which didn't provide the 'birds eye view' we needed. This made debugging and finding issues extremely tedious, and often straight up impossible.
Improving visibility was our primary concern when moving to Google Cloud. After the migration our setup now includes state-of-the-art monitoring and logging! We have insight into everything from failed remote game launches to how many queries our database is receiving. Most importantly though, we know exactly how long certain operations take, using that knowledge we can quickly and accurately track down 'slow code'.
And tracking down 'slow code' is exactly what we did. Before launching our new servers to the public we ran hours upon hours of testing over many weeks to find operations where the code could be faster. So far we've improved the speed of: Getting bookings on the Monitor, viewing the game marketplace, loading experiences in the Operator Panel, and much more!
Our monitoring dashboard on Google Cloud.
Another important fix was an obvious one: Adding more power. Not only did we increase the maximum power of our servers by about ten times but we're also are making use of that power in smarter ways. Our new monitoring tools offers us better ways to measure traffic so we can automatically scale our servers with much more precision. This means that as SpringboardVR continues to grow the application will stay as fast as it is today!
If you are a nerd and want to know more details about how we achieved this speed, you should have a look at our companion post on the SpringboardVR Engineering Blog. I should warn though, it's a very technical post!
Features, and Bug Fixes
While migrating to our purpose built Google Cloud infrastructure and ensuring that the code is efficient and powerful is an enormous step in the right direction, we still have a long way to go to make SpringboardVR the best it can be. With these new tools we can find and fix problems much faster, before they impact your business. This move also lays a foundation where we can deliver new and exciting features to the platform even faster. Check out our roadmap at features.springboardvr.com and if you have any questions feel free to contact us at [email protected]!