I've moved!

Well I decided to move and you can find me here! I am leaving these articles for legacy links but they are also located on my new site.

Writing High Performance Code on the iPhone

Over the last few months a lot of people have asked me how we write our code for the iPhone. Do we use straight C99? Or do we use C++? If so do we use exceptions? RTTI? STL? So I decided that instead of responding by email I would detail it in an article here.

Now please keep in mind that this isn't meant to be a bible on iPhone or mobile/embedded game development. You should always solve the problem with the right tools, and no two games are alike. Would you really try to develop a casual puzzle game using the exact same methods you would use to build a cutting edge shooter?

Also note that the majority of people that have been asking me these questions are PC/Mac developers who are either looking at or have already started iPhone development. So they are the target audience of this article. For the majority of developers who have worked on mobile/embedded devices and game consoles there will likely be nothing new here.

So let's get down to business.

What language did we use to build our iPhone engine?
We use a mix of C and C++ that I believe is commonly referred to as C with classes. The majority of the code would actually be classified as C, but we group objects together in a class (as well as using namespaces) rather than a struct and functions. The decision to allow classes was made simply because it was preferred by the development team, and we knew we would end up having to reimplement a lot of what C++ gives us with objects.

Exception Handling
We have exceptions disabled in the compiler settings for all projects. The current version of GCC used in the iPhone SDK has zero cost exceptions, but you should understand that this does not mean zero overhead. What it means is that there is no execution time overhead when an exception does not occur. Great we have free error handling? Not quite! The compiler will generate additional code and every stack frame will now contain added information required for stack unwinding. With an embedded device like the iPhone the unwind tables will take away resources from the precious little amount that we have. Meanwhile we don't lose much at all by disabling exceptions as there are very few errors that should actually be classified as exceptional on the iPhone. Considering you are shipping for an embedded device, your software has a single point of origin (App Store), and users are unable to modify the file system, the majority of errors will be developer errors. Of course there are some exceptional situations such as low memory, unable to allocate memory, network connection dropped, etc; but for the most part this is a very small group.

We use assertions very heavily in order to catch developer errors, and functions will return error codes in places where that functionality is needed. There are a lot of C++ programmers that heavily rely on exceptions, which is fine 99% of the time on the PC where you have near infinite resources (although this can be debatable), but on an embedded device I need a smaller footprint and all the resources I can get my hands on. To anyone interested in exception handling and wanting to know the cost and implementation details I encourage you to read the Exception Handling ABI for the ARM Architecture.

Virtual Functions, RTTI, and Multiple Inheritance
All of our projects have RTTI disabled in the compiler settings for a few reasons. The first being that while a failed dynamic cast on a pointer will return null, in the case of a reference cast it will throw an exception, which we have disabled. Another reason is that some compiler implementations will end up adding roughly 40 bytes per class of overhead (although many have much better implementations and I have not researched GCC to know exactly how it handles this). The base object model in our engine gives us the non-generic RTTI functionality that we need and only 4 bytes of overhead. Although we do not use this often because as a rule we make sure not to delay any type checking that can take place at compile time rather than run time. This way not only do we have a smaller footprint and faster execution time, but we also have earlier developer error detection.

I'm not going to say much regarding multiple inheritance / virtual inheritance other than we don't use this functionality on any platform and we design our code as not to need it.

Virtual functions are one of the great things that C++ gives us over C, but we use them extremely sparingly. Of course there are the usual rules of not using virtual functions in performance critical areas, tight loops, etc; but we also go a little further. As our engine runs on multiple platforms there is a need for different platform specific subsystems such as rendering. This is easily handled in C++ by creating an interface of virtual functions and then having your platform specific classes inherit from this. But at the same time this is not completely free and on some platforms the performance is extremely bad. You could end up disabling branch prediction and in the worst cases trashing the cache or flushing the instruction pipeline. Dereferencing the pointer may not be the main concern on some platforms, but the compiler skipping optimizations and inlining can be. What we do in these situations is to use platform #defines to include and typedef or inherit. Andre Weissflog from Radon Labs made a great post about this on his blog and how he switched to this method with the Nebula 3 engine. You are sacrificing the abstract class / virtual interface design in this way, but anyone with a PPC device (whether it be Mac, PS3, or 360) will tell you that there is definitely a cost with a missed branch or flush. I did not test the performance impact on an ARM device like the iPhone simply because a small benchmark will show you nothing, and at the time we made this decision we didn't have a large codebase to test with.

Memory Management
This is the one area that can actually differ from regular game console development (at least up until the 360/PS3 but even those systems have better memory gaurantees). In most cases you would do all of the memory management in your engine by allocating various heaps for different uses and making sure to minimize fragmentation. So what makes the iPhone any different?

While consoles have dedicated amounts of memory that you can count on (note that with the 360/PS3 you have to account for the background system usage and events such as the dashboard/XMB which use memory) the iPhone will not always give you what you ask for on application launch. On top of that the iPhoneOS will detect that it is running low on memory and send a warning to all of the current running applications (your game as well as the background applications such as phone, mail, etc) because in the case of a call or SMS there would be the possibility of failure. If you receive this notification you had better free up some resources as soon as possible otherwise the iPhone will force quit your process. With the iPod Touch and iPad this is much less of a concern as there are no phone interruptions, although they will still give you low memory warnings.

This is still an area that we are constantly working on and optimizing. Currently we are using nedmalloc as an allocator due to the extremely low memory fragmentation (we gained a performance boost just by using this rather than the default allocator). On top of that we have various pools for application, level, and frame lifetime allocations as well as pools for specific uses such as particle effects.

I will write another post strictly on this issue in the future once I have more time to analyze the exact performance impact and issues that we are running into.

C++ STL
When developing on the PC or Mac I love the STL because it is simple to use and gives you fully debugged and working containers and algorithms. Unfortunately because our engine and games and built for multiple platforms, the majority of which are embedded devices, we do not use it. Why? Well as I mentioned earlier we have exception handling and RTTI disabled in the compiler settings. Also note that memory management isn't the most fun thing in the world to watch when using the STL on a constrained platform. The STL is a great library for generic cases on a variety of platforms, but for us custom containers were needed.

We did not want to start from scratch and wanted to use containers that were written, debugged, tested, and preferably used in shipped titles. So where could we find this magical library? Again I will point you to the blog of Andre Weissflog where you can download the Nebula 3 SDK which is MIT licensed and use/modify the containers how you see fit. Sure we could have built our own container library from scratch instead, which would have also meant testing and debugging, but time is not infinite and we chose to concentrate on the engine, graphics, and gameplay. This library gave us the functionality we needed with very little modification.

Conclusion
We are using more of a C style approach adding in the convenience of C++ for working with objects. Please don't confuse this with "we used C++ for object oriented programming" because we could have accomplished the exact same design in straight C99. Objects would be structures instead of classes and functions would have a this pointer as the initial argument.

We sacrificed some common C++ idioms, designs, and functionality in order to gain maximum performance on the iPhone and other embedded devices because our games push the hardware to its limit.

I barely scratched the surface with this introductory article and really only covered the very basic details. There are still many more topics to cover such as SIMD using VFP and NEON (you will need this for animation and skinning performance whether you like it or not), various game loop structures on the different devices, optimizing mesh data and packing vertex data with your asset pipeline, memory alignment, using compressed PVRTC textures, and many more. These topics require an in depth explanation and implementation details / samples so I have saved them for a series of future articles.


Also here are some great reference points for console engine development, graphics, and the iPhone:

Oolong Engine - An MIT licensed iPhone game engine owned and managed by Wolfgang Engel, co-founder and CTO of Confetti Special Effects and former lead graphics programmer at Rockstar San Diego.

Diary of a Graphics Programmer - The personal blog of Wolfgang Engel containing extremely good information on graphics programming and iPhone development.

The Brain Dump - The personal blog of Andre Weissflog, co-founder and technical director at Radon Labs. Home of the MIT licensed Nebula 3 engine and a source of very good information on cross platform and console game engine development.

Gamedev.Net OpenGL Forums - Not only do these forums contain years worth of great information on GL and GLES but it is a great community to get help with problems you are unable to solve.

iPhone 3G Rendering Pipeline Video - A video by the lead iPhone developer Renaldas Zioma at Unity Technologies giving an in depth view of the iPhone GPU and drivers.

Good Coding!

Fighting piracy on the app store

It is hard to gauge the exact amount of piracy on the App Store and how it will affect your business model. Some companies have reported an 80-90% piracy rate on their titles, while others have claimed to have a much lower rate. Is there a viable way to deal with this issue? I believe there is within reason.

My company has categorized three types of potential players of our games:

User A
This user legally purchases their games directly from the App Store. They use the App Store featured and top listings to find games they would like, and possibly reading gaming sites such as touchArcade.

User B
This user will pirate some or most of their games from various sources, but they may also purchase titles if they are given further reason. A few of these users may be compelled to purchase your game if they really enjoy it, but you shouldn't count on any substantial revenue from that source. The majority of these users will only make a purchase in two cases:

1. If it is the only possible way to play (i.e. server based games such as MMOs where you control the environment).
   
2. If purchasing the game gives them an added benefit over playing a pirated copy.
   

User C

This user pirates all of their games and applications and refuses to make any purchases. The chance of this user purchasing your game is likely zero and is purely a lost cause.

So what can we do to lower piracy based on these user categories? Well both User A and User C have an easily defined course of action.

User A
In order to have this user purchase your game you need to make a high quality and polished title that appeals to them. Easier said than done! But at least you can rest assured that if you create an appealing game with high production value, this user will make a purchase. Keep in mind that the user needs to find your game somehow, which again is no easy task, but that is a subject for another post and has no direct relation to piracy.

User C
As I stated earlier this user will never purchase your game and therefore you should not concern yourself with them. If they want to pirate your game they will and there is literally nothing you can do about it. Focusing time on this user not only takes resources away from your key focus, but has always proved to be quite fruitless.

Ok. That sounds easy enough, but what about User B?

User B
For now let us assume that your game is not an MMO game, therefore you have no way of completely restricting piracy. It is a single or multi player game that sells on the App Store. How is it possible to convince User B to purchase your game rather than pirate?

What have people done in the past?

1. Detecting whether the game has been pirated (there are numerous methods, all of which can easily be tampered with) and then either exiting the game, disabling features, or popping a message box telling the user this is an illegal copy. This can deter some piracy but for the most part has only resulted in bad press for the game as the user tells their friends "Don't download game X because all it does is crash!". Also the IPA is normally hacked and those checks disabled unless you have littered your codebase with inline calls, etc.
   
   
2. Implemented "call home" functionality so the game needs to connect to a central server in order to authenticate. So you mean I can't play this game on the subway? (which is where most of my time playing iPhone games is spent) What if my internet connection goes out? What if your server goes down? I have yet to see a good system implemented based on this other than Steam and that system has fallbacks and a full support department.
   

Neither of these solutions appealed to me, so the two games my company currently has in production are based on the "value added" model. These were the criteria that I required:

1. The ability to play the single player version of the game wherever you are and at any time. This means a fully featured version with no "call home" functionality, no crashing, and the user experience is not changed by nagging dialogs. Honestly this was our first priority as we want the best experience for User A and we also don't want User C running to the blogs and forums (which happen to be the same sites that User A & B read to find their next purchases!) claiming that the game is buggy or broken.
   
   
2. Create an online aspect for the games that are restricted to legally purchased titles. When a user tries to use an online component our server checks for a valid receipt from Apple. At this point we display a message to the user explaining why they are restricted from using this functionality. Since this component is online we have complete control and it isn't something local that could be cracked.
   

The first point is pretty straight forward. For the single player mode of the games we do absolutely nothing to combat piracy, not even displaying a message box warning the user that we know their little secret. But what about the second point? Sure it sounds great but what could you possibly do? Restricting leaderboards and small features really haven't proven effective in the past.

Our idea was to create a fully integrated online experience rather than just tack on a few small features once the game was completed. My company hasn't yet announced or shown our games, so let's envision a hypothetical game in order to describe our system. As stated above any user (both legally purchased copies as well as pirated) can play the single player version of the game unhindered. One key reason for this is not only to avoid the bad reviews, but the thought that if User B really enjoys the game there is a good chance they will purchase it in order to access the "value added" components.

So what are these magical "value added" components I keep talking about? Here are a few of the things the system provides our games:

1. Online gameplay is either completely run on the server, or works peer to peer with the server acting as the man in the middle.
   
   
2. The game could have items and equipment that is only available in online play. This also opened up the ability to play the single player game in "online mode" with your items.
   
   
3. Each player can create an optional account that works across all of our games. This provides features such as leaderboards, statistics, achievements, friend lists, private messaging, forums, access to tournaments, etc.
   
   
4. The game could have specific online "extras" available to user accounts. Using our hypothetical game a user could upload their 3 hero party to the server and then access it from any device (iPhone, iPod Touch, iPad, Mac, PC). They could also decide to share their party layout with their friends, or even with the entire player base.
   
   
5. Game specific web presence and community. In our example we could have a user gallery where you can see your current party, any parties you have previously saved, and also look at the other player's configurations. People can show off their special items earned in online play, achievements, statistics, etc.
   

There are a lot more points that I will talk about in detail in the future, but currently some are a competitive advantage for us. We realized up front that the majority of these systems were generic enough and if we built them now we could use them in all of our future titles, whether they are App Store games, selling on Steam, or using any other digital distribution method. The cost of hosting the online components is unbelievably cheap these days by using services like Amazon EC2 and Windows Azure. Not to mention the cost scales based on the amount of users who have actually purchased the game.

Conclusion
In conclusion we decided to build a much better user experience for our paying customers where it was possible for us to control access. Even if every one of my assumptions turn out to be incorrect and we don't see an increase in sales from User B, I believe our time was spent wisely. Not only were we able to avoid wasting our time trying to forcefully fight piracy, but we ended up creating a really great online experience for the User A's of the world who have obtained the game legally. We also get the added value that our solution brings to the marketing campaigns and word of mouth.

Of course this model does not fit every single game type, and is most definitely the wrong solution for many different types of games. Although for many of the larger games implementing a model like this will not only increase your chances of higher revenue, but you end up with a much better overall user experience.