The C++ Standard Needs to Broaden Its Reach

I started writing games on computers in 1981. I used a language called Sinclair BASIC on the ZX81 computer, a machine with 1024 bytes of RAM and a monochrome, character-mapped frame buffer that plugged into a domestic TV set. After typing a few dozen lines of BASIC code, I would type “run” and the game would run immediately. After a while I moved onto assembly language and would write a lot of assembly, type something like “assemble”, then type “run”, and the game would run immediately. I moaned to myself about it being a two-step process and came up with a way of using a single command to assemble and run. It made the iteration quicker by about two seconds.

This may seem unbelievable now: the ludicrous idea that, after editing your codebase, you should be able to simply start the game in the time it takes you to breathe out. Yet this rapid iteration lit the fire under the British game development community. I mourn the passing of those days: I don’t remember the last time it took me under a second to get to the front end of my game.

I now write games in C++, and the C++ standard simply tells us about compilation. It demands nothing about the environment in which this takes place, the expectations the user can have about debugging, about the interoperability of compiled pieces of code, or about a host of other things that I could take for granted back in the golden days. C++ needs to evolve and offer more to the user to speed up the development of code.

When I think about the next ten years of game development engineering, I have two visions: the vision I expect, and the vision I want.

The vision I expect involves untrammeled iteration. I see myself sitting at a keyboard, typing code while the game is running and seeing the changes reflected immediately: it is the promise of Edit and Continue finally delivered universally. I hear that my colleagues in art and audio have a similar vision, although its fruition may be nearer for them.

It is unlikely that C++ is going to move beyond its edit-compile-link-debug cycle while I’m still making games; however, users of the Visual Studio IDE will be familiar with the operation of Intellisense, which highlights code errors as you type them. This is powered by a compiler (separate from the Microsoft compiler) that runs in the background while the user edits their source file. The obvious question is this: “Why not compile object files as you type?”

Unfortunately, it’s not that simple: amending include files can create drastic ripples through the code base, requiring such a large amount of recompilation that the promise of always-ready objects seems far away. If we move away from the inclusion-based model and move towards the emerging module-based model of compilation, then perhaps this will in fact be achievable. Modules promise a much better specified library unit.

The vision I want is slightly different and also revolves around the shortcomings of the C++ ecosystem. I regard the package options of other languages with envy. For example, the Python Package Index makes reinventing the wheel completely unnecessary: it is a repository exposing a bounty of pre-built third-party libraries trivially available for inclusion in your own projects.

This is not feasible in C++. A library built by MSVC is not interoperable with a library built with GCC. Even a collection of libraries built by MSVC are not necessarily interoperable: some command line switches create ABI conflicts, such as those dealing with exception handling, iterator debug level, runtime, or static library bindings. Even something as simple as a pre-processor definition can create an ABI conflict.

As a result, third-party code can only be imported under very particular circumstances. Either the library is supplied as header-only and simply informs compilation in total, or it is supplied as source plus headers so that compilation switches can be chosen as necessary, or it is supplied as headers plus a library file and the consuming developer hopes for the best.

In the case of Visual Studio and Windows-targeted development, we have had a long period of ABI stability, since the 2015 edition. I remember the times when upgrading your version of Visual Studio meant trawling through your third-party binaries and locating versions matching the new ABI. If you subscribe to the MSVC standard library GitHub comments, then you will know that there is an ABI break on the horizon for Visual Studio, and I fear it is going to be much more unpleasant than before. Unfortunately, the problem with the first two options is the concomitant increase in build times. If the ecosystem is expanded to support agnostic third-party distribution of binaries, then this will all go away.

This matters. Even as long ago as 20 years, when I was working on Rome: Total War, building the game took a considerable amount of time. In fact, it took long enough for me to get out of my chair, put on my coat, leave the building, amble to the convenience store, buy a doughnut, amble back to the studio, make a pot of tea, let it brew, drink it while eating my doughnut, and completing a Guardian crossword before the build was finished. Possibly not enough time for the cryptic crossword though. This had a terrible impact on bug-fixing: fixing one bug at a time was costly, and you would expect to batch bug fixes together at the cost of separation of concerns, since you would only expect to run a build three or four times a day.

This vision I want is not so far-fetched though.

Imagine being able to simply swap binaries in and out of your project, a linear algebra library here, an AI library there, a ray-tracing library, a match-making library. The process of creating games from fundamental building blocks becomes much easier without all that tedious mucking about with configurations. It is a democratizing effect, lowering the expertise needed to develop viable games.

There is a will in the C++ standard committee to tackle this problem, and the embryonic form of a standard C++ ecosystem is visible in the proposal. Interoperability is the great reward if only we can properly define it.

The C++ Standards Committee is an entirely voluntary body, driven by people who turn up. Please consider sending delegations from your studio to steer C++ in a direction favorable to game development. Mine is the only voice currently: being one of two hundred is not as effective as being thirty of two hundred. My DMs are open if you want to participate in effecting this change.