Yesterday, I decided to explore what a sufficiently-motivated developer could do to improve content build times in XNA Game Studio 4.0. In a relatively short time, I was able to hack together a solution that distributes content builds across available processors, and doesn’t lock up the IDE (as described here). This means Visual Studio remains responsive while your content builds, you can track build progress in the Output Window, and the whole thing will finish faster.
Before continuing, let me state clearly that what I’m about to describe requires Visual Studio 2010 Professional. Those of you using Visual C# Express Edition won’t be able to use this technique.
Okay, now I’m going to explain what I did. In my sample solution, I reduced the build time by 25% on my dual-core laptop (from 38 seconds down to 29 seconds).
For my experiment, I started with the Ship Game starter kit from create.msdn.com. I chose this starter kit because it contained enough content that locking up the IDE on rebuilds would get annoying really fast.
Here’s what I did (short version; longer version at the end):
- Split content into multiple content projects, all building to the same output directory.
- Add a Game Library project to the solution that references all the content projects.
- Add a VC++ Makefile project to the solution that builds the game library from (2) with multi-proc support enabled.
The key here is that I introduced a VC++ Makefile project to initiate the content build. This project type invokes arbitrary, user-defined commands, and it isn’t all broken like the C# projects. Specifically, it doesn’t lock up the UI thread when it executes long-running commands, and it correctly redirects all the command output to the Output Window, and reports errors in the Error List.
To have the makefile project build all my content projects in parallel, I set up the Build command to build the Game Library project using msbuild.exe with multi-processor support enabled.
Example Build Command:
“$(MSBuildBinPath)\msbuild.exe” /m:$(NUMBER_OF_PROCESSORS) “$(SolutionDir)Content_All\Content_All.csproj”
For Rebuild and Clean, I simply added /t:Rebuild and /t:Clean, respectively.
Note: VC# Express doesn’t support VC++ projects, which is why this won’t work for Express users.
So that’s what I did, and it works great. Content builds don’t lock up the IDE, and the Output Window displays the name of each asset as it is compiled (some reassuring feedback during long-running builds).
At this stage, my solution is set up to build content projects referenced by the Game Library project (Content_All) in parallel. When I tried it the first time, I only squeezed 2 seconds off the build time. The bottleneck in my build was the project using the normal mapping processor, so I added another project, and moved some of the models and dependencies over there. That brought more parallelism into the build and the overall build time dropped to 29 seconds (from 38).
I experimented a little with putting the audio in another project, and shuffling some of the models back and forth, but gave up before finding any additional gains. On a machine with more processors (my laptop has two), there is potential for more savings.
I should note that in order to split up the custom model processing into multiple projects, I had to include some of their dependencies twice. For example, both projects build the NormalMapping shader. The cost of the redundancy was offset by a greater savings from parallel processing.
The drawbacks to doing this are:
- You basically need two extra projects in your solution to enable it, for each target platform.
- You need to manually edit the Solution Configurations to make things build properly.
- For Windows projects, the content won’t be recognized by ClickOnce publishing.
The first isn’t so bad, because you can hide the extra projects in a Solution Folder. The second is something many people don’t really understand, but should be part of any advanced developer’s tool kit.
Another thing I want to note is that I separated the assets that use the custom processor from the assets that use standard processors to reduce incremental build times when NormalMappingModelProcessor.dll is modified. The standard content won’t rebuild at all – and thanks to splitting the custom content into two projects, the custom content can rebuild fully in less time as well.
Angry grumbling: I had intended to provide my modified ShipGame solution as a downloadable sample, because the web site said it was under Ms-Pl. However, the actual ZIP file contains a license file that describes the “XNA Premium Content” license, which prohibits redistribution of unmodified software.
Before I go, let me leave you with this disclaimer: your mileage may vary. I do not plan to write out a click-by-click tutorial, but I added more detail below.
Here’s what I did (longer version):
- Move ShipGame content project up to the solution folder, and rename it Content_Main.
- Add new content project, named Content_Custom.
- Delete the Content Reference from ShipGameWindows.
- Add new Game Library project to the solution, named Content_All.
- Change the output folder of Content_All to match the output folder of ShipGameWindows.
- In Content_All, add references to Content_Main and Content_Custom.
- Change the Content Root Directory of all content projects to “Content”.
- Add new C++ Makefile project, named Content_BuildAllParallel, and configure the Build, Rebuild, and Clean commands (details below).
- Move all the content that depends on NormalMappingModelProcessor into Content_Custom, along with any additional content it depends on.
- In Content_Custom, add a reference to NormalMappingModelProcessor.
- In Content_Main, remove the reference to NormalMappingModelProcessor.
- Edit the solution configuration so that Content_All and NormalMappingModelProcessor are excluded from all solution builds. Make sure that Content_BuildAllParallel is built when the rest of the game is built.