Baking Ambient Occlusion Maps for Simple Objects : Advanced options

As discussed in the previous section, The Basics of Baking Ambient Occlusion Maps, it was shown that using the Texture Bake system to generate Ambient Occlusion maps requires relatively little effort. There are times however, when more complex objects or maps are needed, and doing this isn’t always as simple or straightforward as it might at first seem; due not to complications with the tools themselves, but rather as a result of needing a more advanced awareness and understanding of the broader universal principles behind all forms of texture baking, although in this particular instance applicable to ambient occlusion.

The following discusses some of these points individually and what they generally mean to the process of baking ambient occlusion textures.

Texture/UVW Map Space

Texture baking in Blender is determined by a number of critical prerequisites that centre around an objects “UVW map” and “Texture” relationship, i.e. the interplay between their position and placement relative to each other and the areas that relationship defines. In affect, the rendering process associated with texture baking is looking at the objects surface data represented by the UVW maps as a ‘template’ of sorts, a guide into which different types of image data can be written. The accumulative effect of this texture/UVW map relationship is that, 1) the area occupied by the texture is the objects “Texture Space“; 2) the area defined by the border its “Texture Bounds“; 3) the space within this area is ‘unique’, written to, and used (generally) on, a 1:1 basis.

In practice, because UVW’s are so fundamentally important to proper texture baking, it’s vital meshes are correctly mapped beforehand; objects should be unwrapped so UVW’s fall strictly within available texture space because the process can only write data to that area, UVW’s that extend beyond this are ignored and left un rendered.

Texture Overlap & Shared UVW’s

Texture baking is ostensibly about “uniqueness“, at a fundamental level the process uses the individual characteristics of a mesh and renders those features to an equally individual image. Because of this fact, that baked textures are inherently ‘unique’ in character, it presents another critical prerequisite, that UVW maps should be as ‘clean’ as possible. In practice this means UVW’s should be unwrapped so no overlapping occurs (shared or otherwise obstructed areas), this prevents problem areas being rendered several time, once for each UVW ‘layer’. If on the other hand some form of shared UVW is required, for example the leg of the tutorial chair being duplicated around the mesh instead of using individually modelled sections, then care needs to be taken to position elements correctly within the overall layout to avoid the aforementioned rendering problem. See also “Mirrored Meshes” below for more details on this.

Ambient Occlusion & Joining Meshes

As mentioned briefly above, ambient occlusion shading is determined by the relationships objects and surfaces have to each other, the closer one object is to another, the darker the immediate area surround each due to the way in which their proximity occludes and/or attenuates ambient light. Because this happens it means that objects may need to be collated together before baking so the collective ‘group’ is properly representative of the final state in which the model is to be used.

In practice this typically means; 1) separate objects and/or elements should be joined together into a single contiguous mesh. 2) elements should be positioned relative to final use, with the caveat that 3) if an object reuses sections they should be placed, but not joined, to the parent mesh so their presence can influence the render, i.e. their location should be such that they have an effect on ambient shading as that relates directly to their immediate and localised proximity to other items instead of being pertinent to the element they’re duplicated from – if making repeat use of a single leg in place of all for example, that leg should be an instanced duplication of an original (the original still being attached to the main mesh), these duplicates are then regarded as positional in nature only and inherently temporary – they directly affect the occlusion of their immediate surroundings but NOT that of leg itself.

Ambient Occlusion & Object Proximity

The relationship between an object and the scene in which it’s placed is another aspect of baking texture maps that is of notable importance because it directly affects how items are illuminated by ambient lighting; as with joining meshes mentioned above, a models position should be directly reflective of its eventual use in game. For example, if the tutorial chair were to be used solely within the context of being backed up against a wall, in Blender the scene would need to be similarly representative of this so ambient occlusion could be correctly baked relative to that specific use/scenario – a series of mesh primitives (flat plains or cubes) would need to be placed into the 3D view as representative of ‘floors’, ‘walls’ and other such surfaces. Shown below for example, ambient occlusion has been baked with and without a floor surface to show the difference between the situational specific shading both create whereby the chair legs near the ground are correctly shaded relative to being placed feet first on the ground when the mesh primitive is present and visa-versa when it’s not.

This also holds true if the chair is to be used in other contexts, being knocked over for example, which results in occlusion shading that differs a great deal from the upright stance, it might be necessary to bake another map to cater for that orientation. It’s not usually necessary to go to the ‘nth’ degree however, baking a separate ambient occlusion map for every degree of rotation or position, as generally speaking simple ‘generic’ maps that indicate typical behaviours are all that should be necessary.

Material Colour & Ambient Occlusion Influence

As mentioned briefly in the previous chapter using the ‘Normalized‘ setting when baking AO maps forces Blender to ignore any Diffuse colours assigned to Materials. This in turn ensures ambient occlusion shading is rendered to image maps in ‘true’ greyscale, showing no colour influence or tinting. There is a caveat to doing this however, using the setting tends to produce maps that are typically quite high in contrast; the difference between ‘black’ shading and ‘white’ highlights is often unnecessarily stark. To alleviate this, Normalized can be de-activated but this does then mean the ‘Diffuse’ colour has to be set to ‘white‘ (RGB 1.000) or some other neutral ‘grey‘ tone to compensate (where each “R“, “G” and “B” value is the same to ensure an ‘equalised’ tonal value). That is of course if a greyscale ambient occlusion map is required. Being able to change the colour of the Diffuse to ‘white’ also means that said-same can be changed to any colour, and doing that means tinting the resulting bake with whatever colour is then set in the diffuse slot, ‘blue’ for example as shown below. This can be useful in certain circumstances when ambient occlusion needs to match the general tone of a texture or other asset.

Texture Bake & Anti-aliasing

The rendering mechanism used to bake texture maps is not fully part of, or has limited access to, Blenders main rendering features, chief amongst which for the purpose of texture baking is “Anti-Aliasing“. In practice this means all types of texture bake, ambient occlusion included, exhibit the same associated problem, the appearance of a ‘stepped‘ or ‘jagged‘ border around non-axial UVW’s edges – any lines that deviate from a straight or perpendicular path is prone to this problem (shown below). Whilst it can be elevated to a certain degree by setting a ‘Margin‘ value higher than the default “2” this may exacerbate the problem by causing a related issue, that of “texture bleed“, which is especially problematic where an objects UVW has been tightly mapped. In both instances – using margin and/or having a tightly packed layout – it’s best to over-compensate for the Anti-Aliasing limitation by positioning UVW elements in such a way as to allow more room for margin bleed where necessary, with curves and organic UVW’s needing particular attention paid.

Mirrored & Split Meshes

The most troublesome types of object to render can be those containing sections that are either mirrored or duplicates of other areas of a mesh. They present a number of particular problems because only half, or incomplete sections of a mesh are being used to render a full map. As ambient occlusion is generally about ‘uniqueness’, any form of duplication tends to make achieving this slightly more tricky than normal because meshes have to be set up and rendered in such a way as to prevent the associated side-effects from being inadvertently baked, such as join seams, centre-lines, shading artefacts, misplaced or misaligned shading and so on. The way around these potential issues is to bake the ambient occlusion maps from objects that are left intact, once done they can then be split, duplicated and/or mirrored where appropriate. It’s best done this way to facilitate the proper processing and calculation of a given objects ambient occlusion shading based on it being lit as a solid and complete object – if only half the object is present it’s baked relative to that, in other words, Blender doesn’t know it’s only processing a partial object.

Mesh Smoothing & Ambient Occlusion

The two main attributes of rendering a successful ambient occlusion maps are, 1) the physical structure of the object, and 2) any Mesh Smoothing or smooth groups applied therein. This is because both affect the way objects are lit; the former through the physical attenuation of ambient illumination; the latter by altering the relationship between faces through the control and interplay of ‘edges’ and ‘regions’ so they appear either ‘hard’ or ‘soft’ to the viewer. Although smoothing can be assigned to meshes in a number of ways (“Edge Split” modifier, “Split” faces, mesh detachment or separation) each essentially performs the same fundamental function, that of splitting vertices apart to form regions which have separate, but coincidental vertex associations – where ever these vertex splits occur surface continuity, and as a result illumination, is broken, causing an edge to be formed in the mesh which is in turn baked into the map. In practice this means it’s important to ensure that mesh smoothing (and/or smooth groups) have been properly applied or assigned before rendering so ambient occlusion shading is correctly representative of the object being baked and its general surfaces properties, i.e. ‘hard’ or ‘soft’ surfaces or edges.

Mesh Tessellation (Triangulation)

Meshes that have complex shapes, typically usually curved and non-axial, tend to exhibit a problem when baked whereby shading isn’t properly aligned with respect to the part of the mesh they’re supposed to be from. This is due to the way in which Blender interprets quadratic faces. As they can be flipped along their internal diagonal in one of two directions, occasionally Blender gets this wrong and bakes the quads ambient occlusion ‘left’ instead of ‘right’, ‘up’ instead of ‘down’. In practice this means it may be necessary for an object to be “triangulated” (“tessellated“) using “Ctrl+T” before it’s baked to force Blender to correctly interpret face orientation (this may also involve re-organising the orientation of faces so they are rotated correctly relative to being ‘concave’ or ‘convex’ shapes).

Problems Baking Ambient Occlusion

Although using Texture Bake is usually straightforward it is not without issue, exacerbated by the version of Blender being used – it may be that a particular error or problem is present in one version but not another so it is always worth checking results against official builds. The following then are common problems that happen generally agnostic of Blender version.

Nothing happens: Make sure the object to be baked is selected and ‘active’, rendering is object sensitive, i.e. it only works when an item is selected for use.

Black maps/nothing was rendered: Make sure object has a texture assigned to the UVW map. When objects contain more than one texture map make sure ambient occlusion is being baked to the right one. Additionally, check Material colour assignments and/or that “Normalized” is active.

Black spots or missing areas: Check the UVW map to make sure the black or missing areas are not the result of the bake process re-rendering over previously processed areas or that parts of the UVW map are outside the texture bounds.

Excessive jagged edges: Check texture resolution. The lack of anti-aliasing can be reduced by using textures of a larger size for render baking and then resizing those down as required – the anti-aliasing available in a photo editing suite will further reduce the problems associated with there being no aliasing.

Ambient Occlusion is flipped/upside down: Check the objects orientation and/or that the various “Apply” modes haven’t been set on a mirrored mesh section which can invert or flip associated data.