Topic: Normal maps: texture space and orientation

[kat]

An updated version of this article is available here »


Texture optimising and normal maps
If you're starting to get into the creation of normal maps for your own or other mod projects then here's a little tip that you may not have thought about, and it might explain why you get odd results some times where you don't expect them.

In the past when making assets for game based on lightmapped or vertex lit rendering engines, artists have been concerned with 'texture space', that is, trying to squeeze as much out of a given image size as possible because the orientation of the image didn't matter; applying textures to brush faces or models allowed level designers and modellers to simply rotate a texture in pace so it faced the right direction.

Enter nextGen and dynamic 'bumpmapped' lighting
If you carry over your texture techniques into the new dynamically lit engines however, creating 2D artwork from which you need to render normal maps needs a bit of thought to get round a problem that you don't initially think of when your head is buried in Photoshop et-al.

Take a look at the image below, it's a height template that's going to be used to generate a normal map for application to a model; the model happens to be a stone pillar. In order to get as good a texture space usage as possible out of a single 512x512 texture the artist (me) has gone ahead with their traditional techniques and positioned of the two sections so a 'split' runs down the centre of the image; this allows a good tiling vs texture density ratio; the 'bricks' of the pillar on the left; the 'trim/base' on the right.

If you run this through a 2D to normal map converter (I used the GIMP's normal map filter for this example) you'll get the image below, there's nothing overtly wrong with that image on initial inspection until you start to think about something; which way is 'up'?!.

Because of the way normal maps are implemented by game engines, rendering the 'bump' effect in game means the engine gets it directional data (which way the texture is supposed to be facing) from the colours present in the normal map; depending on how those colours are being read, the game will render up/down, left/right and height/depth accordingly.

So what's the problem?
There's a problem here with the original template, the trim section has been rotated so it's positioned to get the best tiling and texture density ratio possible on a 512x512. This now means that the trim section is effectively physically lying on it's side, the result of this being that the normal map converter will rendered the colour orientation according to that position.

If you were put this into game as is, in spite of rotating the texture in a level design editor or 3D modelling application, you'd find the brick body displaying correctly in game; shadows and highlights behave correctly. Looking at the base trim however, you're likely to find at least one of the 'axis' inverted with the result that the object is rendering shadows and highlights 'backwards' - it's showing highlights when it should be showing shadows.

It's doing this because the colour orientation of the normal map rendered from the template is incorrect for the trim section. As mentioned above the image was lying on it's side when rendered to normal map so the lighting in game looks at the normal map based on that; in effect one (or more) of the axis's are flipped; lying on it's side like this means the top/bottom axis is actually left/right as the normal map converter see's it.

How to fix
In this particular instance the fix can't be done by flipping individual channels until the texture shows up correctly; because only a section of the image is wrong, trying to correct it that way means 'breaking' the sections that were previously OK.

What needs to be done instead is for the original template image to be split into two separate images, for the trim to then be rotated so it's aligned to the correct orientation for what it represents, and then for both to be rendered to normal maps separately. This now means that both sections are rendered according to the correct interpretation of 'up' so when they show up in game the light/shadows render relative to the normal map.