It’s been about a week and a half since my last update. I’ve completely re-written my entity system so that it’s more in line with established conventions for these things, and so that it is somewhat simplified. I’m almost back to where I was before in terms of basic functionality. But the system is more flexible and I feel better about where it’s going.
Some of the articles that have proven useful are:
- Entity Systems are the future of MMOG development
- Dungeon Siege architecture
- Artemis Entity System Framework
- Documentation for the Unity game engine
- A Dynamic Component Architecture for High Performance Gameplay (GDC 2010)
- State of the Art Game Objects
- Game Engines 101 – The Entity/Component Model
- What is an Entity Framework
Some of the articles are frustratingly high level and/or theoretical, so that is why I always prefer information that comes from shipped games such as Dungeon Siege.
At a high level, what I have now is roughly similar to what I had 2 weeks ago in that it is basically generic game objects that have data attached that determines their behavior. So let’s go over the differences.
One of the main differences is that I’m now using the formalized “Component” concept to store data for Entities (I’ll call them Entities now instead of GameObjects). Before, I was using generic property bags.
So for instance, before I would declare that a Torch object would have a property bag that consists of: Position, Orientation, Visible, Lit (and maybe a few other properties). In addition, I had somewhat of an adhoc binding declared in code that said “for the Torch template, use this 3d model, this particle system, and this light source for the light”.
Now I have switched to Components. Components are, in a way, groups of properties that belong together. For instance, there is a Placement component which has Position and Orientation. There is a Flammable component which indicates how something burns. There is an Aspect component that indicates which 3d model to use, and whether an entity is visible. So, a torch might consist of these three components. Note that now I express the 3d model in data instead of code. This is sort of orthogonal to Properties vs Components but came out of the same work.
Fundamentally, Components are just like properties, but less granular. I still don’t have enough experience with this to be absolutely convinced that less granularity is better here. Reduced granularity perhaps helps with organizing things logically (e.g. snap these three blocks together and I have a working object, as opposed to having to pick through all the properties I need), but I think a pure property bag system could work too.
Components are a little more work when it comes to serialization. I am now writing custom persistence logic for each Component type instead of just per property type. However, this does let me customize the persistence logic (for instance to store a float value at a lower precision if high precision is not necessary).
My GameObjects are now renamed Entities. Some articles argue that entities should just be a number (a globally unique id). I do currently use a class for each entity for convenience, but I could easily switch that completely to an id system. Entities do have a bunch of state associated with them that doesn’t exist as data in their Components. This state is:
- UniqueId – 32 bit integer that identifies the entity uniquely in the entire game world
- LiveId – 32 bit integer that is unique only among current active entities, and serves as an index into some arrays for fast lookup
- OwnerUniqueId – the UniqueId of this entity’s owner (e.g. if its in someone’s inventory, or in a chest or something), or it could be the id of a physical area of the game world
- TemplateId – 32 bit integer that identifies the template this entity came from. More on this later
- ComponentTypesBitField – a bit field that serves as a quick way to identify which Components are attached to the entity (yes, this limits me to 32 or 64 types of components).
Another big difference is that I’m now using the formalized System concept. Systems operate over Entities that have a given set of Components. For instance, the RenderingSystem requires that an Entity have Placement and Aspect Components. Anytime an Entity is added that has those two components, it also gets added to the RenderingSystem.
Systems are an abstraction that I think nicely addresses the problem of inter-component dependencies. If you do a survey of the literature for game entity frameworks, you’ll often find critiques of the approach center around communication between different components (e.g. the “ModelRendering” component needs to know about the “Position” component). These approaches also tend to have a bunch of logic in the Components.
In the approach I’m using, all the logic is in the Systems. The Components are just data. Direct communication between components is unnecessary, as logic that requires this is in the System which already knows about the Components its interested in.
Another difference from my previous iteration is that I support Entity templates. This is done in a very similar manner to Dungeon Siege’s templates (but less flexible). Entities are instantiated based on a given template. Furthermore, when saving an entity to persisted storage we check which Components of that Entity have different data than that expressed in its template, and we only store those Components. When the Entity is re-hydrated, we fill in any missing Components from its template. Of course, each Component knows how to clone itself.
I have the templates currently defined in a text file, so I could even support adding new templates at runtime in the editor if I wished. Currently I just parse the file at startup though.
I’ve implemented a system based on the GDC 2010 article mentioned above that allows the following actions to be done in constant time with just an array lookup (not even a hashtable/dictionary lookup):
- Get an entity by its liveId (uniqueId still requires a dictionary lookup)
- Add an entity or its components
- Remove an entity or its components
- Add/remove an entity from a system
This does require a bit of extra memory to maintain all the lookup tables though. I’m not sure in the end whether it will be better than just doing Dictionary lookups, so this may have been premature optimization.
A task I’m currently engaged in is figuring out which component types I need. While one of the benefits of Component-based design is that there is flexibility to add new types of things as I work on the game, I really need to ensure that the main component types are correctly factored up front.
If you take a look at the Dungeon Siege architecture article, some of their component factoring seems like a mess. For instance, the Aspect component, which describes the visual qualities of an object, also contains health and mana properties and a list of chores. That seems like a poor factoring of responsibility.
Perhaps in a few weeks I’ll be more confident about how I’ve organized things and I’ll provide more details.
One of the big problems I had was how to persist entities. This is needed for save games, but also simply for activating and deactivating entities as the player moves through seamless open world. When you leave an area, its entities are essentially removed from all systems and “put to sleep” by serializing them to a byte stream. Persistence concerns are something that none of the entity system articles I link to above ever mention.
An entity could have a given set of static state at the beginning of the game. Then, it could be modified during game play, changing its state. In addition some state is more important to persist than others, depending on the location of the player. For instance, knowing whether an NPC is dead is very important – this state should obviously be saved. However, saving the NPCs position is not important if the NPC is in a far away part of the world. The NPC’s position would be recalculated based on whatever tasks/schedules he is performing when the player enters that part of the world. For NPCs that are currently in the active zone though, we do want to store their position. Otherwise, they would “jump” when the player saved/reloaded the game.
I thought of various ways to tackle this problem, but essentially it always boiled down to having up to 3 layers of state that we would need to look at: Load state for the current active region (NPC’s position). Ok, what am I missing? Dig through the persisted state for that region and find it (NPC’s health). Ok, what am I missing? Dig through the static state for that region (NPC’s name).
This gets even more complex when you consider that game objects may span different regions during their lifetime. An NPC could theoretically move from one region to another (I don’t have explicit plans for that, but I wouldn’t want to design myself into a corner). So now I really can’t store “per region” static state about an object. It all needs to be global. That means having a giant global list of objects in some serialized form, but through which I can quickly jump to a random location and re-hydrate some state I need.
This seemed to be getting more and more complex, so I made some cuts.
First, I decided that static state could be eliminated (in a sense). If save game size becomes an issue, I can revisit later. But for now, I have made that decision. In some sense, Entity templates can take some of the weight off this – much of an entity’s state might just be part of its template.
Second, I decided that I can ignore the difference between “important” persisted state (NPC’s health), and “transient” persisted state (NPC’s position). I’ll just persist it all. I don’t think there any many cases where this is an interesting distinction anyway.
So with those changes, now I can just serialize the entity as one blob in one place. The starting state of the game will essentially just be an initial “save game”. (In addition, this also gets me a rough idea of how big save games could get without having to play through the whole game.)
So I plan to have persisted data for each region which contains the entities that exist in that region. In addition there will be a set of global state that represents things such as the player’s inventory and such.
What about modifying persisted objects? Do I need to find what region they are in, re-hydrate the entities from that region, modify the specified entity’s state, and dehydrate them again? This is where reading the established literature came in handy. In one of the articles, they mentioned addressing this by only allowing modification of dehydrated entities via messages. For instance, you could send a message that expresses “Add ‘foo’ to the conversation tree for John-the-NPC”. If John-the-NPC is currently hydrated, then great, it is delivered and the state is changed. If John-the-NPC is dehydrated, we simply queue the message up. Whenever a region is re-hydrated, we simply try delivering all the queued-up messages again. Of course, this means that the messages must also be persisted to the save game state.
I can’t really say much about performance at this point. I’ll really only know if it ends up being good/bad when my game is done with the full varied set of components.
One clear benefit to a design like this is better usage of the instruction and data caches. It is set up such that you run the same set of logic on the same type of data at one time.
Unfortunately, C#, unlike C++, lacks the ability to use custom heaps and allocators and placement new. These would all help with ensuring data that is accessed together lies in memory together. If all my Components were structs (value types) instead of classes then this is somewhat alleviated. Unfortunately you can’t hand out a references to a struct to a caller in C# – they’re always returned by value. So using structs, while probably possible, would place a lot of restrictions on my code. So for now I use classes for Components.
Luckily, objects allocated in sequence are allocated in adjacent memory in .net (assuming there is room). So by pre-allocating my Component arrays I should at least be able to ensure that all the Components of one type are allocated in contiguous memory. Hopefully that should retain much of the theoretical performance benefit.
Here is a thread with a lot of horrible advice about component based systems, especially around performance. Posters are giving performance advice based on tests with limited test data sets, and they are reporting performance results in frames per second!
This has just kind of been a brain dump of where I’m at currently. I could provide some nice graphics and tables that explain more, but I think the linked articles do that well enough.