Modifying shaders at runtime

I added this feature to my engine a few weeks ago. This lets me modify my shader in visual studio and see the results in my game within a few seconds, instead of needing to stop, recompile and restart the game. This is a feature most real game engines have, so it makes sense that I would too.

I figured it would be pretty useful, but I wasn’t prepared for just how useful!

First of all, it makes it easy/fun to tweak shader parameters. Adjusting the lighting or colors of things is almost no effort. This can encourage you to test different looks and come up with something that looks a lot nicer that it would have otherwise.

Second, it provides a very useful debugging mechanism. Something doesn’t look right in your terrain renderer? Maybe one of your lighting calculations is off. Make a quick change to directly output the suspicious value from the shader, and see the result a few seconds later. Just in the week or so where I’ve had this working, I’ve debugged so many issues like this. It doesn’t replace things like PIX, but it provides a convenient form of “printf” debugging in a way.

As an example, here is some water:

Maybe I need to see what it looks like if I flatten the normals. One line change to the shader and a second later:

Maybe I suspect something is wrong with my noise modulation for the flow. Let’s visualize that:

Ok, now let’s visualize water depth:

All these things are possible in just a couple of seconds, with no forethought.

How does it work?

I use, as a base, the content building sample from Microsoft’s XNA education catalog. Except instead of allowing for changing Models at runtime, I allow for changing Effects (it’s generic enough that it wouldn’t be too much extra work to have this work for textures or other resources).

A background thread spins up that periodically scans the original content folder for changes to any files whose name matches assets that have been loaded by the custom ContentManager (HotSwapContentManager) Yes, you need to replace the default ContentManager with this one. It’s a bit hacked together, so you probably don’t want to use this in production code, so only enable this in your debug build, or “game editor” build, or whatever. I’m sure there are still bugs.

When you load an Effect from the HotSwapContentManager, you need to wrap it in a SwappableEffectWrapper. This wrapper will contain the most recent version of the effect – as you make changes to the original effect file and save it, it will be rebuilt, loaded through the content pipeline and placed in the wrapper.

More importantly, I make an attempt to copy all EffectParameter values from the old Effect to the new one. There may still be some bugs in this code – it works just well enough for my purposes. There are still some unimplemented EffectParameter types, and when encountered these will throw a NotImplementedException. You can then add code to handle those cases.

The usage of a wrapper isn’t strictly necessary for getting something like this to work. I could have instead just had the ContentManager hand out the newest one when asked (and instead of always fetching the Effect from the wrapper, you would always refetch it from the ContentManager prior to use). In fact, this would simply things a bit (I did it this way because in my scenario I need to handle Effect subclasses. So if you have an Effect subclass, it will recreate a new instance of that subclass).

Sample project

Here is the link to the project.

To use it, add a reference to SwappableShaderLibrary to your game. Create an instance of HotSwapContentManager and specify where your content source files are with respect to your project’s output directory. Then, add HotSwapContentManager to your game’s Component’s list (in your Initialize method, prior to base.Initialize()). For instance:

// Set up hotswap content manager and assign to the game's components.
hotSwapContentManager = new HotSwapContentManager(this.Services, @"..\..\..\..\SwappableShaderContent");
hotSwapContentManager.RootDirectory = "Content";

Load any Effects using this content manager, and put them into a SwappableEffectWrapper:

effectWrapper = new SwappableEffectWrapper<Effect>(hotSwapContentManager.Load<Effect>(@"MyEffect"));

and fetch the Effect from the wrapper every time prior to using it:

spriteBatch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend, null, null, null, effectWrapper.Effect);
 spriteBatch.Draw(texture, new Rectangle(0, 0, GraphicsDevice.PresentationParameters.BackBufferWidth, GraphicsDevice.PresentationParameters.BackBufferHeight), Color.White);

You don’t need to copy any EffectParameters over, but you will need to set the CurrentTechnique again if you had changed it.


5 comments on “Modifying shaders at runtime

  1. We use hotloading in our game/editor too. I found it useful to wrap the methods and properties of the native resource (eg Texture2D), as well as having an implicit cast operator in the wrapper class that basically just returns a reference to the native resource. This allows us to use the wrapper like the native resource. It’s very easy to replace the wrapper with the native resource and vica versa at any time this way.

    We also used the ContentBuilder class to rebuild assets, but ran into the problen that our asset settings were lost. With asset settings I mean the stuff you setup in VS, eg if a texture should be built as DXT. Now we just use the .contentproj files of our project and msbuild to rebuild all assets.

  2. Sounds like you have a more complete system in place :-). I’ll probably want to support it for other resources like Texture2Ds eventually, so I’ll need to know the asset settings too. Good tip there.

    • Yes, it’s pretty complete actually and support custom types as well. If you’re interested in our solution, feel free to drop me an email and I’ll send you the corresponding .cs files.

  3. Instead of spinning a thread that checks the content dir periodically, you could use FileSystemWatcher (http://msdn.microsoft.com/library/system.io.filesystemwatcher.aspx). It notifies when a file or a directory has changed, so you can refresh only then. That way your “content building thread” would consume less CPU time.

    This is the way I’ve done hot-swapping in my engine.

    • Yeah, I started with FileSystemWatcher, but it didn’t appear to work at all. I never succeeded in any getting any event from it (and I’m not the only one, judging from my google searches on the topic).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Just another WordPress site

Just another WordPress.com site

Harebrained Schemes

Developer's blog for IceFall Games

kosmonaut's blog

3d GFX and more


Turn up the rez!

bitsquid: development blog

Developer's blog for IceFall Games

Game Development by Sean

Developer's blog for IceFall Games

Lost Garden

Developer's blog for IceFall Games


Developer's blog for IceFall Games

Casey Muratori's Blog

Developer's blog for IceFall Games


Developer's blog for IceFall Games

Rendering Evolution

Developer's blog for IceFall Games

Simon schreibt.

Developer's blog for IceFall Games

Dev & Techno-phage

Do Computers Dream of Electric Developper?

- Woolfe -

Developer's blog for IceFall Games

Fabio Ferrara

Game Developer

Clone of Duty: Stonehenge

First Person Shooter coming soon to the XBOX 360

Low Tide Productions

Games and other artsy stuff...


Just another WordPress.com site

Sipty's Writing

Take a look inside the mind of a game developer.

%d bloggers like this: