<story><title>How the EVE Online Servers Deal with a 3,000 Person Battle</title><url>http://penny-arcade.com/report/editorial-article/planning-for-war-how-the-eve-online-servers-deal-with-a-3000-person-battle</url></story><parent_chain><item><author>jere</author><text>[Edit: this comment turned out not to really be about Eve and more about the difficulty of real time MMOs]&lt;p&gt;This blows my mind. I&apos;ve been thinking about writing a small multiplayer game for a few weeks. And today I was actually doing back of the napkin calculations. The thing that quickly became clear is bandwidth is O(n^2) where n is the number of players in the same location, since you have to share the location and velocity of each player with each other player.&lt;p&gt;Here&apos;s an example of a calculation I was using. It&apos;s the monthly bandwidth in TB of having a certain amount of players in a shared space (3000 in this case), assuming that the location/orientation/velocity of each player&apos;s ship is contained in a measly 48 bytes and you attempt 30fps.&lt;p&gt;(48 * 30 * 3000 * 3000 * 3600 * 24 * 30) / 1e12 = 33592 TB = 33 PB&lt;p&gt;THIRTY THREE PETA BYTES to simulate a single battle for a month.&lt;p&gt;I&apos;m looking around at various VPS providers and the bandwidth they offer. Using the naive calculation above and the cheapest Linode offering and it&apos;d take $3 million/month to support a single one of these ongoing battles.&lt;p&gt;Now, I&apos;m sure the numbers above aren&apos;t used in reality. Obviously, you can reduce the updates per second... but not by much. You can&apos;t shave much off how much you send. It at least gives you a sense of the orders of magnitude required to support such a thing. Even contemplating supporting 100-1000 people at once is looking very difficult for me to pull off.</text></item></parent_chain><comment><author>hythloday</author><text>Hey, I used to be a games networking programmer, and your estimate is pretty good. A few corrections that bring it down to something not entirely unaffordable:&lt;p&gt;* You can bit-pack your structures a lot more efficiently - let&apos;s say 2 bytes for a local player ID, 10 bytes for each of velocity/orientation (quaternion represention and heavily quantizing the theta component), and 10 bytes for the location. Clients are never allowed to determine the canonical physics simulation, so all of the above are really just for display purposes, and can be trimmed down as appopriate - we don&apos;t need to worry about desyncing (as I&apos;ll explain in a bit)&lt;p&gt;* 30 packets-per-second is way too high for network play, with a game of EVE&apos;s mechanics you could probably get away with something perhaps as low as 1 pps. Intermediate simulation of the player entities is done by a technique called dead reckoning that&apos;s linked earlier (though in practice you&apos;d use a slight improvement on it to stop entities leaping around the world).&lt;p&gt;* Sometimes game mechanics allow you to strongly cut down the number packets you send. For example, in EVE, it might be desirable to not send information about a ship if it&apos;s completely occluded by another ship, or to only send information about ships that are in a frontal cone ahead of you. This usually doesn&apos;t affect the bandwidth function (though sometimes it does), but you can almost always cut a constant factor of (rule of thumb) 50% off the bill.&lt;p&gt;So that ends up being 32 * 1 * 3000 * 3000 ~= 250MB/s, which looks about right to me. One thing you didn&apos;t account for is that you don&apos;t typically allow MMO game clients to connect to each other, but to multiplex everything through a server. So it ends up being twice as large as that - 500MB/s.&lt;p&gt;Good luck with your game! 100 interacting players in an MMO is a challenging target but not an unreachable one for a single-developer game, and it&apos;s definitely a very interesting project to undertake.</text></comment>
<story><title>How the EVE Online Servers Deal with a 3,000 Person Battle</title><url>http://penny-arcade.com/report/editorial-article/planning-for-war-how-the-eve-online-servers-deal-with-a-3000-person-battle</url></story><parent_chain><item><author>jere</author><text>[Edit: this comment turned out not to really be about Eve and more about the difficulty of real time MMOs]&lt;p&gt;This blows my mind. I&apos;ve been thinking about writing a small multiplayer game for a few weeks. And today I was actually doing back of the napkin calculations. The thing that quickly became clear is bandwidth is O(n^2) where n is the number of players in the same location, since you have to share the location and velocity of each player with each other player.&lt;p&gt;Here&apos;s an example of a calculation I was using. It&apos;s the monthly bandwidth in TB of having a certain amount of players in a shared space (3000 in this case), assuming that the location/orientation/velocity of each player&apos;s ship is contained in a measly 48 bytes and you attempt 30fps.&lt;p&gt;(48 * 30 * 3000 * 3000 * 3600 * 24 * 30) / 1e12 = 33592 TB = 33 PB&lt;p&gt;THIRTY THREE PETA BYTES to simulate a single battle for a month.&lt;p&gt;I&apos;m looking around at various VPS providers and the bandwidth they offer. Using the naive calculation above and the cheapest Linode offering and it&apos;d take $3 million/month to support a single one of these ongoing battles.&lt;p&gt;Now, I&apos;m sure the numbers above aren&apos;t used in reality. Obviously, you can reduce the updates per second... but not by much. You can&apos;t shave much off how much you send. It at least gives you a sense of the orders of magnitude required to support such a thing. Even contemplating supporting 100-1000 people at once is looking very difficult for me to pull off.</text></item></parent_chain><comment><author>EarthLaunch</author><text>I&apos;m a game developer bootstrapping an Erlang-based game server that attempts this.&lt;p&gt;You don&apos;t have to send 48 bytes each frame (you can usually send only delta), and 30/sec is only necessary in a reaction-based FPS type of game. Hosting players in a local area, local meaning they can see each other and interact, is n^2. My calculations a year ago came up with a per-minute cost for the amount of AWS hardware it would require to host 1,000 and 10,000 &apos;local&apos; players. Basically, it wouldn&apos;t be affordable to have a huge gathering/battle more than a few times a month. It would be VERY expensive. Plus, each player has to have enough download bandwidth.&lt;p&gt;This type of thing will definitely happen in the future. Clustering is still in its infancy, and it&apos;s picking up steam. And there are smart optimizations to be made, such as sending more frequent data about nearer players. You can also design around it by not making players usually local to each other even though there is a single seamless game world. I&apos;m doing that.</text></comment>