Twitter  Facebook  Google+  YouTube  E-Mail  RSS
The One Man MMO Project
The story of a lone developer's quest to build an online world :: MMO programming, design, and industry commentary
Distributing Your Game via CDN - Not Just for Megacorps Anymore
By Robert Basler on 2015-04-13 10:18:28
Homepage: email:one at onemanmmo dot com

When I started developing The Imperial Realm :: Miranda, Content Delivery Networks (CDNs) were big and fancy and expensive and only used by giant multinationals so I never imagined I'd be able to afford to use one.

With me posting more and more big screenshots on my websites like this one:

[Random screenshot of the week.]

Last weekend I decided it was time to research solutions to improve my websites' performance.

Initially I was thinking I needed to rent a web server, but then I came across CloudFlare which for $20 a month offers to speed up your website by cacheing static website content on nice big fast servers. My non-static data is pretty small, I have a lot of big bitmaps I'm serving, I want that to be fast, so a solution that didn't need a new web server sounded pretty attractive. (I've rented web servers before, and they have all been problematic. Sometimes another website on the server is using all the CPU, or somebody misconfigures their CMS resulting in a breakin, or the thing is just down and nobody knows why or can tell you when it will be fixed.) Doing my due-diligence on CloudFlare, I found a few worrying stories about downtime, but even more worrying was a benchmark that showed CloudFlare sometimes having several hundred milliseconds of latency serving requests. Amazon CloudFront was on the same graph with very low latency across the board.

While reading Amazon's website, I came across information on their S3 service. S3 is basically a really stupid web server. What interested me about S3 though, is that it has smart tools for uploading files.

At a guesstimate, it looks like Miranda is going to clock in at around 100GB of data. Luckily, most people will only need to download the client which is pretty small and a small portion of that 100GB. But I still have to manage the full 100GB which is a very large and unwieldy chunk of data. At that size, just getting the data to the web server is a big problem. My reasonably fast internet connection will take about 100 hours to upload that (I'll likely upgrade to internet that can do that in about twelve and a half hours.) Until fibre appeared in my front yard earlier this year, I figured I'd have to take the data to some data centre on a hard drive whenever I wanted to do an update.

But S3 has a better way. They have a tool which will compare your local files with hashes of the files in the server bucket so you can upload only the files that have changed. The terrain data in Miranda is pretty stable, so it will be rare that I need to upload all 100GB of Miranda.

The problem then is that S3 isn't designed to support lots of people downloading lots of files. Luckily, you can point CloudFront at an S3 bucket to get fast performance and smart file management all in one package!

So to learn how Amazon's services work, I signed up for CloudFront and set up,, and to use it. That was surprisingly easy, all I needed to do was add a CNAME record for each domain and point that at a CloudFront URL then change all the static data links to point to the new subdomain. Easy peasy. went from about 7 seconds to load to under 1 on the website speed test I used.

At current levels, all three domains will be free for the first year.

The cacheing which makes CloudFront fast, makes distributing Miranda a little more complicated. If I simply update a file in the game, it may take a while for the changed file to be distributed to all the CDN servers, so for a time, it could be serving both the old and the new versions of that file, which doesn't really work for Miranda. CDN's do have a feature to invalidate files so that you can be sure everybody is on the same version, but it takes minutes not seconds, and it is a cost and management headache. Fortunately there's a better option: versioning.

If you put a version number as part of the URL, then as soon as clients start requesting that version, it will be instantly and automatically propagated to all the CDN servers.

When I designed Miranda's HTTP file request system I built it with the idea that I'd need to have a number of web servers to support the demand of thousands of players. To do that, the client can pull files from several web servers simultaneously and distribute the load. Using a CDN, as far as the client is concerned, there's only one server so all of that isn't strictly necessary anymore. The change I did need was to add support for inserting a version number into the URL's that the game requests.

With distribution at long last figured out, next up is to figure out what to do about the game server.

By the way, I also did due diligence on Amazon as well. Whatever you do, don't upload your Amazon credentials to Github.

New Comment

Cookie Warning

We were unable to retrieve our session cookie from your web browser. If pressing F5 once to reload this page does not get rid of this message, please read this to learn more.

You will not be able to post until you resolve this problem.

Comment (You can use HTML, but please double-check web link URLs and HTML tags!)
Your Name
Homepage (optional, don't include http://)
Email (optional, but automatically spam protected so please do)
Plastic or paper? (What's this?)

  Admin Log In

[The Imperial Realm :: Miranda] [Blog] [Gallery] [About]
Terms Of Use & Privacy Policy