🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

What’s the best way of saving/loading game data within an up to date object graph?

Started by
8 comments, last by arnero 4 years, 6 months ago

Hi All,

I’m looking for some advice and best practices in handling serialization/deserialization of objects for use in an ARPG game I’m looking to write at some stage in C#. Specifically, this question relates to saving data and re-loading that data. This isn't game engine specific.

Scenario: I have a class that presents a player (Player). Within that class, there is a property that stores the character’s role playing class (CharacterClass). The CharacterClass class describes a name and some minimum and maximum values for character attributes.

public class Player
{
       public string Name { get; set; }

       public long Experience { get; set; }

       public CharacterClass CharacterClass { get; set; }

       ...
}

public class CharacterClass
{
       public string Name { get; set; }

       public MinMaxValue Strength { get; set; }

       public MinMaxValue Dexterity { get; set; }

       ...
}

When I save this information via XML or JSON serialization, I’m saving the object graph/state as it is at the time. I can then load this at the game start-up to restore part of the player’s saved game.

{  
   "Name":"Arthur",
   "Experience":0,
   "CharacterClass":{  
      "Name":"Mage",
      "Strength":{  
         "Min":10,
         "Max":80
      },
      "Dexterity":{  
         "Min":15,
         "Max":90
      }
   }
}

The problem: What if the property values within CharacterClass changes? Suppose the name changes or the character class attribute values change. The data for the CharacterClass class will be loaded from a file at game start-up separately but the disconnected data from the player save game file no longer contains an up-to-date version of the object.

This same concept applies to a lot of Game Objects that have child classes as properties. Another example could include player equipment with affixes, suffixes, rarity, etc. Each one of these is either an object or list of objects. If all these values are serialized, they may not represent the current data held in the master data store separate from the player save game data.

Another concern discounting the data state is the size. Serializing all connected objects in the parent object saved would likely take up a massive amount of space and is wasting memory when deserializing. All these values could be loaded in to one list at startup and resolved/referenced from there.

Potential solution: I could mark the CharacterClass property inside the Player class as ignored so it isn’t serialized and add a reference identifier string on the CharacterClass class that is placed inside the Player class like so:

public string CharacterClassId { get; set; }

[IgnoreDataMember]
public CharacterClass CharacterClass { get; set; }

During object initialization I could look-up the reference string in a list/store of CharacterClass (e.g. in memory list) and re-connect the object to the Player object. This would be an up to date version of the object. The resulting JSON would look like:

{  
   "Name":"Arthur",
   "Experience":0,
   "CharacterClassId":"MageClass001"
}

My concern with this solution is the amount of manual work required to re-connect disconnected objects during initialization.

Questions:

  1. Is this solution viable?
  2. Is there a better way of doing this?
  3. Does anyone know how this is handled as an industry standard or in your own games and if so could they point me to any relevant documentation?

Dave

Advertisement

Sharepoint XML files and AWS YML files are full of references similar to your "CharacterClassId". Some of these Ids consist of gibberish GUIDs filling my complete screen horizontally.

>manual work required to re-connect disconnected objects during initialization
Id is a special type. You probably could hook somewhere into Newtonsoft.JSON
for reconnection all your Ids.

Also your title says "Graph". Not "Tree". In JSON you show a Tree. How do you handle cycles? WindowsInstallerXml for example is full of id Attributes and ref Attributes (local to the file). I think office open documents make it a habit that all references are pointing at a different XML file within the ZIP archive.

  1. Is this solution viable?
  2. Is there a better way of doing this?
  3. Does anyone know how this is handled as an industry standard or in your own games and if so could they point me to any relevant documentation?

1) It can work. While the game you have in development is constantly changing, the game you give to customers is static. If/when you update you will need to migrate the data structures. (Back in the era where games shipped on cartridges, cards, or disks that couldn't be updated, no migration was necessary in the final products.)

2) Build the system designed to be self-updating. More on that in answer #3.

3) It depends entirely on the games and the tools they use, but there are some common patterns.

One option is versioned structures. ALWAYS include a version number in your data structures. Provide an update path that goes all the way back to the first version. Sometimes that means updating from v22 to v23. Sometimes that means v1-->v2-->v3-->...v22-->v23. If you add or remove fields, you need to deal with that. Often this means a multi-step loading, read a first token to know the data type, then in the data type read a token to know the version number, then load the number of bytes that correspond to that version, then update if necessary from the old version number to the current version number.

Another option is to always store name/value pairs, which can work in conjunction with versions as a stored name/value pair. Provide defaults in case the values aren't there. For example, a data stream class might have a family of reading functions: bool readBool( s32 key, bool& value, bool& default); bool readS32( s32 key, s32& value, s32& default); bool readString( s32 key, string& value, string& default); Often this is done in conjunction with version numbers, littered with code like: if(version<14) { /* read three values and process them to new format */ } 

frob said:

v1-->v2-->v3-->...v22-->v23.

That looks like Git. Or ASP.NET core entity framework. Like!

In my experience, you need to understand the boundary between your dynamic data - the information that is created at the start of a play session and changed through play - and static data - the information that is created by the developer and which is only changed through updates to the game and patches. The static data exists in a pre-serialised form, and you wouldn't serialise the static data out with a typical save game, but would instead just reference it. These references can use IDs or UUIDs instead of names to make them less brittle in the face of updates. Or you can use a more complex and permanent naming scheme.

There are of course some complications at the boundaries - e.g. if you decided you wanted to remove a character class entirely, or if you changed a value in the static data that otherwise invalidates the dynamic data (e.g. a character class MaxHealth might reduce by some amount for game balance purposes, but your character's CurrentHealth is now too high). This is similar to a data schema migration in the traditional technology world, and is typically handled via an explicit migration mechanism - e.g. the patch that changes the the MaxHealth value is also responsible for either changing the other data to match it, OR for introducing code that handles the disparity at load time.

Kylotan said:

In my experience, you need to understand the boundary between your dynamic data - the information that is created at the start of a play session and changed through play - and static data - the information that is created by the developer and which is only changed through updates to the game and patches. The static data exists in a pre-serialised form, and you wouldn't serialise the static data out with a typical save game, but would instead just reference it. These references can use IDs or UUIDs instead of names to make them less brittle in the face of updates. Or you can use a more complex and permanent naming scheme.

There are of course some complications at the boundaries - e.g. if you decided you wanted to remove a character class entirely, or if you changed a value in the static data that otherwise invalidates the dynamic data (e.g. a character class MaxHealth might reduce by some amount for game balance purposes, but your character's CurrentHealth is now too high). This is similar to a data schema migration in the traditional technology world, and is typically handled via an explicit migration mechanism - e.g. the patch that changes the the MaxHealth value is also responsible for either changing the other data to match it, OR for introducing code that handles the disparity at load time.

@Kylotan Understood on the difference between static data and dynamic data. The idea to reference the static game data in the serialised save data using an Id/UUID's to point to the the static data seems sound to me. It's a case of automating this process during load of save data in a programmatic reusable manner. I don't like the concept of reconnecting each reference to it's pre-loaded static data manually in code.

I may be able to create a DataAnnotation or similar structure to identify referenced fields in the save data (a string or Guid) and reload the data in to a paired complex field (e.g. an object) that isn't serialised to save data.

Also thanks for touching on issues that will arise from changing/removing existing data. It's something I need to think about during patching/updates. Mentioned by @frob, versioning the structures seems like a good way to solve this problem and having appropriate migration methods.

@arnero In terms of Netwtonsoft JSON, do you know if the library supports re-connecting saved Ids (string/Guid) to an instance of an existing object? I'll research this over the next few hours.

Thank you all for your responses. This is something (making a game) I've always wanted to do and this has been a blocker in my mind for a while. I'm a line-of-business application programmer by trade and these issues don't typically arise in applications I develop (e.g. data mapping is handled by Entity Framework).

A generic serialisation approach can sometimes work to eliminate redundancy by spotting repeated elements, but it will always lack the context to know whether the objects in the graph are transient or long-lasting, constant or mutable. So, if you want to use a generic serializer, you would need to consider how this metadata can make it into this system, e.g.:

  • Some sort of [StaticData] attribute on static data which the serialiser can respond to (and again, work
  • StaticReference<Type> with custom serialisation functions, instead of referring to Type directly
  • Custom serialisation functions on the static data classes to write placeholder IDs instead of the actual contents

I think you spotted most of these in your original post.

It's worth bearing in mind that game developers have had to store game state long before generic serialisation tools and libraries were available to us, so there's a long history of writing custom save/load code which only reads and writes precisely what we explicitly ask for. As you mentioned, it's a lot of manual work, but it only gets done once, for a relatively small number of types, so it's not usually a heavy burden.

Sorry, I just read the ASP.NET core has two JSON serializers. One lightweight and the NewtonSoft thing with all bells and whistles. It is open source, so ..

This topic is closed to new replies.

Advertisement