:: The story of a lone developer's quest to build an online world
:: MMO programming, design, and industry commentary
[The Imperial Realm :: Miranda] [Blog] [Gallery] [About]

The One Man MMO Project: Stuck!

By Robert Basler on 2012-02-23 02:15:12
Homepage: www.onemanmmo.com email:one at onemanmmo dot com

One morning when my daughter was very little, I was shaken awake by a very frustrated little girl. Opening my eyes, she was pulling and pulling on her little arm, which was completely underneath me. Seeing I was awake, she looked up at me, straight in the eye, and despite having only recently started to talk, she opened her mouth and clear as a bell, she said "STUCK!!!" Today I can relate to how she felt.

A couple days ago I was reading the November Game Developer magazine, always a source of good stuff. There was an article The Game Entity by Michael A.Carr-Robb-John. I have an entity system in my game, so I was interested to see what he had to say. Now there was a lot of good stuff in there, but one thing really stuck out for me. He mentioned that it is nice for game entities to have variable update frequencies so that entities that nobody is particularly interested in don't burn a lot of CPU cycles. Now for me, expecting to have hundreds of thousands of entities, each with maybe a dozen components being simulated on a single server, this was a brilliant idea, one that could save me a fortune in server costs!

Add to that, he mentioned briefly about the potential of multithreaded updating of entity systems to more effectively use newer CPU's. Another server capacity bonanza.

So Tuesday I started looking at my entity system to see what I would need to do to support multithreading and variable update rates for my entities.

Looking at the entities, only a couple have updates that do anything. (While I was in there, I noticed that a couple other components could have their updates removed really easily. Those updates are really just one-time state checks that could just as easily be done in the functions that set the state - so that's a little win.) But once I started thinking about extending the time between updates for entities which nobody is particularly interested in, I realized that for things like AI and health, if I allow long updates, then the simulation for those entities is going to be wildly inaccurate.

If you only update the health every 30 seconds, if that guy is dead after 1, for 29 seconds the guys attacking him are going to be waiting around instead of moving on to do something else. So updating 1 time per second is much better than 10 times per second, but I'm not gonna be able to get 1 time every 30 seconds that really saves me the big bucks. Ok, I guess I can live with a 10-fold capacity improvement. But is it still worth the trouble of building entities that support variable update rates while maintaining a reasonably accurate simulation? I'm not convinced. One thing I am sure of is that most of the periodic polling in the entity system can be removed and replaced with callbacks and a message-driven design. But message designs are so much more painful and annoying than simply calling member functions to do stuff.

So that got me onto multithreading. The article mentions that multithreading really needs to be built in from the start and talks about using a three phase entity update process with read, execute and write phases. Googling about I didn't really find much helpful information at all about this techique, but I did run across an article from Intel's Smoke demo I read a while back.

Smoke includes an interesting technique where they have a publish-subscribe system for component updates. So if a component is interested in the health value of another component, whenever that health value changes, it can subscribe to get an asynchronous, wait-free update message. There are some refinements on the main idea to improve performance -- you can read about those in the State Manager section of the article. While some entities are interested in other entities, looking at my entities, the majority of component updates for each entity come from other components of that same entity.

So it was Tuesday afternoon, and I decided to go for a walk and get some fresh air. I can think and walk.

While thinking about all those updates going on within entities I considered parallelizing component updates on a per-entity basis and adding a read/write lock to each entity. (Read/write locks are 132 bytes each, so I have to use them sparingly.) Simple per-entity locks are reasonably straightforward, but read/write locks give you the added benefit of letting multiple people read simultaneously while still protecting writes.

Components in my game are accessed by pointers. I ask the System for a pointer to the component of that type for a particular entity, and off I go. This starts to get messy if you accidentally write to a component you only have a read lock on. I spent a few minutes trying to think of automated (or really easy) ways to catch errors with pointers that have read or write permissions. And ways to improve performance for updates within an entity. And ways to make sure those locks get released should anything go wrong. And are read-write locks overkill?

Getting home and looking at all the code that accesses the entity system, there's already a lot of code I'd have to redo to make sure it is read/write safe. Am I already too late to add threading?

Yuck.

Dinner time, end of the work day. 8 hours burned, grinding on a problem, nothing really to show for it.

Today I put that problem aside. I'll let my brain grind on it while I'm asleep, or waiting for my daughter to clean her teeth. Instead I started working on a cool shield effect by adding support for cubemaps to the renderer.

I can't afford days spent grinding. With my goals, every day must see progress. When I get stuck, I have to work harder to put things aside sooner rather than later.

By omeg on 2012-02-28 16:41:44
Homepage: omeg.pl email:
Heh, I've been stuck on proper entity system design myself for a long time. It's still not finished (need to implement templates and easy instancing) but it's close.
Asynchronous messaging is a bit easier in C# but still... My ES (modified Artemis) implements systems that "subscribe" to component types they are interested in. In the main update loop systems get only those entities that have such components. It also has systems updated with various frequencies and parallel processing capabilities, but it's not that easy. I still need to account for entities being modified by more than one system at a time (unlikely, but can lead to "fun" bugs).
By Robert Basler on 2012-02-28 17:14:27
Homepage: www.onemanmmo.com email:one at onemanmmo dot com
Artemis is interesting. It is a variation on what I have. I put more brains in the components, where Systems mostly just iterate through components to have them do their thing although I have a couple systems with more complicated logic. I'll have to think about the value of making components dumber.

The parallel processing is really where I'm stuck. There's just no way I've come up with to do everything either without locks or with automated locking, because manual locking is so prone to impossible to reproduce bugs. Networking is bad enough for hard-to-find bugs. One networking bug on Need for Speed took me 58 attempts to reproduce and it depended on packets getting randomly delivered late.

New Comment

Cookie Warning

We were unable to retrieve our session cookie from your web browser. If reloading this page does not get rid of this message, please read this to learn more.

You will not be able to post until you resolve this problem.

Comment (You can use HTML, but please double-check web link URLs and HTML tags!)
Your Name
Homepage (optional, don't include http://)
Email (optional, but automatically spam protected so please do)
Multiply: 2 and 6 = (What's this?)

Admin Log In