Bake Tiling (tileable) Normal maps from meshes in Blender - Extras

The process of creating highly detailed meshes to bake tiling normal maps is largely the same for non-tiling normal maps, those typically assigned to specific models or scenarios. Where they do differ it's largely to account for tiling versus single-use, rather than what each map is meant to represent visually as a texture (rock, brick, tech surface etc.).

With this in mind the following "extras" section of the "Bake Tiling (tileable) Normal maps from meshes in Blender" discusses some of the idiosyncrasies of generating tileable normal maps using Blender 3D.

Preface - Extras ^

There are two generally usable approaches to creating normal maps, one uses a specially prepared image, the other, models. Whilst the underlying processes differ significantly, functional normal maps can be produced from either when due care and attention is paid to the outcome, and how normal maps are to be used. With that said however, the latter, using models and a like-for-like translation of structure to pixel data, should be considered the more effective way to represent three-dimensional structure within a normal map. Simply put, converted images cannot account for multi-axis topology, the physical realities of height and depth, because they are a single channel (monochrome), single axis interpretation of a flat projection.

Design note: in essence normal maps generated from images represent general surface detail like roughness, pitting, noise, veins and so on, whereas normal maps baked from meshes better represent structure, depth, shape and form.

Mesh vs pixel density ^

The quality of normal maps produced by baking highly detailed meshes is dependant on two aspects of the process; 1) the size of the final image, and 2) the density of the mesh being baked. Although as a general rule of thumb "more is better", there is a point at which including significant detail becomes redundant due to the hard limitations presented by the physical size of the image being baked to. For example, baking a high-resolution mesh that's essentially 1000x1000 polygons across to a 256x256 pixel image equates to there being a disproportionate amount of polygon detail relative to the individual pixels of the bitmap. In essence because each pixel can only be a single normalised RGB colour value, there is potential for significant structural overuse and/or inability for surface detailing to be properly captured to image - the mesh density is too high for the available pixels. In other words, there is a physical limit to the amount of information a high-resolution sculpt can contain before its essentially 'over-saturating' the bake process, much of which is then 'lost' to the final output.

Design note: the RGB normalised colour values represented by each pixel is averaged from mesh data falling within their respective texel (texture pixel) boundaries when mapped and their immediate surroundings.

For smaller normal maps then, high-resolution detailing can be kept to a minimum with preference given to representing major or easily discernable structure, as the texture size increase so too can the amount of detail.

Surface vs structural detail ^

Allied to the above point, consideration should also be given to surface versus structural detail. Here the amount of surface detail represented by the high-resolution mesh will be similarly determined by the eventual size of the baked texture, the larger the better. Dealing with smaller assets then, present similar issues to those raised above, namely detail loss. One especially significant issue of a sculpt having too much surface detail when baked to smaller image assets is the increased potential for introducing 'noise' or 'scatter' as individual pixels average more coarsely the less texture space available.

Design note: as with 'mesh vs pixel density', texels are similarly effected by 'surface vs structural' detailing issues; the larger the texel, the more loss will occur due to increasing amounts of surface data having to be considered and reduced to a single RBG value per image pixel.

In these instances preference should be given to similarly sculpting high detail that emphasises major or easily discernable structure, individual stone slabs rather than incidental or surface detailing like cracks, pits, scratching and so on.

X/Y tiling & Offset ^

For organic and other nonlinear structures like rock etc., care should be taken to ensure the sculpted mesh tiles correctly on the "X" and "Y" axes, that is side-to-side and top-to-bottom ("Z" is generally unused for normal maps). If this is not done the resulting normal map won't tile cleanly and will exhibit hard edges and lines at the textures border, or colour biases or other aberrations not easily corrected or removed using an image or photo-editor. The solution to this is to enable "Tiling:" and "Tile Offset:" during the sculpt process.

Design note: each side of the mesh tends to exhibit a colour bias based on the saturation level of either 'green' and/or 'red' - 100% colour saturation to 0% represented by 'black' (depending on the image editor each colour of the R, G and B colour channels may be displayed tinted the colour of the channel, i.e., green, or grey-scale), which combined with the 'blue' results in the normalised RGB colour values typical of a normal maps. The bias essentially gradients (fades) the saturation, making it more difficult to correct than a simple hard edge.

Once "Sculpt Mode" is active, scroll down the Tool Shelf to the "Tiling:" and "Tile Offset:" sections. Under "Tiling:" left-click the "X" and "Y" buttons to enable the feature, any marks or strokes will then tile appropriately using the Grids global "X" and "Y" axes/centre-lines (the coloured lines that converge at grid "0,0,0"). Once set, for "Tiling:" to then make sense, in "Tile Offset:", if the size of the underlying low-resolution mesh is 2x2 Blender units, the offset values should be set to match, "X: 2.00" and "Y: 2.00" ("Z" not necessary). Together the two settings now mean an active threshold is established at which point the mesh will tile, sculpt actions performed on one side will duplicate, 'tile', across to the other.

Design note: the default Cube, and usually any similar mesh object added to the Scene, defaults to 2x2 units across and centred on the grid, 1 unit per axis quad (8 in all for the default cube object, one 'unit' on each "+" and "-" side of an axis). For the tiling feature to work correctly all meshes need to be similarly placed, i.e., equidistant across the cardinal "X" and "Y" axis, with a limited amount of overspill or 'bleed' in the high resolution to make sure sculpted strokes are properly orientated relative to the offset edge - similar to issues with Smoothing, hard edges can appear when tiling strictly at the offset boundary hence the need to overcompensate slightly to ensue mesh/faces normals are properly aligned relative to their neighbours either side of the threshold. It's also important that, to ensure offset works correctly, both low and high-resolution meshes be 'reset', prior to any sculpting in particular - select each in turn and press "Ctrl+A" selecting "Rotation and Scale" ("Location" is implied). In addition, using fixed units, or increments therein, aid offset precision. In other words, if the low-resolution is 2 x 2 units, the high-resolution can be 2.5 x 2.5 units.

Displace Modifier ^

When creating the high-resolution mesh for baking it's generally best to sculpt or manually shape a subdivided object. For sculpts that need to conform to a preexisting asset this process can be challenging if the end result needs to be a like-for-like representation. A solution to this is to use the "Displace" Modifier to alter the basic structure of the mesh before detailing, that way at least the initial starting point will replicate the desired goal, even if approximately.

Design note: the general point of using an image to displace the mesh is to create a starting point for sculpting that broadly approximates the objects larger structures instead of using it as a short-cut to the finalised sculpt, there generally aren't any.

To do this first create a "Texture" with an "Image" slot carrying the bitmap to be used as the displacement map (this does not expressly need to be attributed to a Material but being so has no detrimental effect) - in "Texture" Properties, select the empty entry in the Texture List aperture and click the "+ New" button. In the now populated panel, scroll down to the "Image" sub-section and left-click "Open". In the "File Browser" that appears, find and select the image to load, then click the "Open" button top-right. Blender will return to the previous screen loading the image into the vacant Image slot. With this now available it can be assigned to the Modifier.

Design note: when clicking "Texture" Properties a "Brush" drop-down menu may be visible instead of the usual texture slot aperture. If this is the case, directly above the "Brush" menu are two buttons, one depicting a 'globe', the other a 'checker'. Click the the 'globe' - when Sculpt Mode has been or is active, Texture Properties tends to default to 'brush' rather than 'surface' options.

Next, select the mesh then in "Modifiers" Properties assign a "Displace" instance (second column from the right under "Deform"). Modifiers Properties will update. In the panel now available left-click "Display modifier in Edit Mode" to activate the modifier so its effects can be seen in Edit Mode (cube icon with vertex selection highlights). Finally "Tab" into Edit Mode selecting the face, all vertices or edges, then click "Subdivide" button in the Tool Shelf a number of times (six at most). The mesh will subdivide and displace based on the image assigned to the modifier. To adjust the influence of the displacement, in the modifiers properties adjust the "Strength:" value, lower to decrease, higher to increase.

Design note: when subdividing keep the initial iterations low, six or less, to prevent the mesh 'spiking' because it's too dense. As an alternative to using the "Subdivide" button to divide the mesh a "Subdivision Surface" Modifier ("SubD", "SubSurf") can be assigned to the Object with a similarly increased density/iterations setting (tops out at "6") - when using this approach activate "Display modifier in Edit Mode" and make sure the modifier instance is at the top of the stack (click the "Move Modifier up in the stack" button to move up the stack).

Once the mesh is deformed to an appropriately degree the modifier should then be applied, readying the mesh for further sculpting - left-click the "Apply" button at the top of the "Displace" modifier (this removes the modifier).

Temporary Image overlay ^

To augment the above the sculpt process can be aided using a simplified version of the diffuse image the normal map represents. Here, prior to the high-resolution mesh being significantly subdivided or shaped, it can be UV unwrapped and assigned a 'helper' graphic that indicates where differing structure should be sculpted - this can be the image used to displace the mesh, where the Displace modifier is used, or a more detailed version of the same. In either case with the 3D View display mode set to "Texture" the image will be visible whilst sculpting, making it easier shape the mesh to fit the texture. For this, in Edit Mode simply select the entire mesh and Unwrap, "U » Unwrap". Then in the UV/Image Editor assigned the image, either the displacement image or another more detailed version - click "Image » Open", browse and select the image, then select and assign to the UV map in the editor once available.

Design note: the image used does not need to be too detailed, similar to displace its purpose to 'guide' the sculpt process. Be mindful that as the mesh is sculpted, it, the UV map, and assigned image will distort - if this happens to sufficient degree it will result in a mismatch between normal map and diffuse when used in game if not corrected beforehand.

Cycles & Tiling Normal maps ^

When using Cycles Render to bake tiling normal maps, although final output is still rendered to an image mapped to the low-resolution mesh, material nodes are used to facilitate this instead of Blenders default material system (Blender Render).

Design note: some of the options discussed below may not be available, or differ slightly from those explained depending upon the version of Blender being used. More detailed information on baking normal maps using Cycles can be found here.

First, in the main "Info" Header running along the top of the application click the "Engine to use for rendering" selector and choose "Cycles Render". Next select the mesh in the 3D View and access the "Node Editor" by switching one of the available areas using the "Editor type..." selector (or use "Shift+F2"). In the Node Editor Header click the "+ New" button to create a new material. A default 'node set' will appear, "Diffuse BSDF" and "Material Output". Next, from the "Add" menu again in the Header, select "Texture » Image Texture". Another node will appear in the work zone, left-click to place. This creates the basic three-node set-up for a basic working material.

Design note: this simple node set-up is roughly equivalent to the various stages a standard Blender Render material would normally have, i.e., "Material", "Texture" and "Image" slots.

Switch to the "UV/Image Editor" and from its Header select "Image » New Image" or click the "+ New" button visible. In the "New Image" properties pop-up that appears change "Name:", alter the "Width:" and "Height:" of the image, and set "Generated Type:" to "UV Grid" or "Color Grid". Click "OK". This creates an assignable image block. Switch back to the Node Editor and left-click the "Browse Image to be linked" drop-down menu of the "Image Texture" node, selecting the image just created from the list that appears. The material will update.

Design note: an image needs to be available for assignment, which can be created before or during node set-up. Preference should be to use a generated image rather than a preexisting bitmap, ostensibly to avoid the "Circular reference in Texture Stack" error.

With the material nodes done, in the "Bake" subsection of "Render" Properties set "Bake Type:" to "Normal", "Space:" to "Tangent", with "Selected to Active" enabled. In the 3D View select the high then low resolution meshes (selection order is important) then and click the "Bake" button. Blender will process the Scene outputting the render to the UV/Image Editor ready for saving.

Design note: when setting up Cycles properties for baking normal maps, "Swizzle:" can be used to optionally change the orientation of each "X", "Y" and "Z" axis - this flips the respective channels to account for some engine technologies inverting different axes. And also "Ray Distance:". Changing this value modifies the distance between mesh surface and the point of origin from which 'rays' are cast to calculate their orientation, position and the general structure - organic structures typically require higher values then non-organic to account for depth variation.

Conclusion ^

Being able to make highly detailed, high-resolution meshes does rely on using equipment adequate to the task. Hardware needs to be able to cope with various levels detailing that can reach into the millions of polygons or triangles. This does not necessarily mean the use of older or slower computers prevents the production of workable normal maps, rather that the process has to be rethought slightly to compensate, to the extent that sculpting becomes an exercise in effectiveness, working within the limitations presented rather than being held to the demands of hyper-realism or detailing. This might mean sculpting and baking major features to a normal maps that are then combined with an image-based conversions to add surface detailing otherwise not possible to place due to hardware constraints.

Design note: although there are techniques and features within Blender that ease the burden of high-resolution work, "Alt+B" to "Clip Select" for example, being able to manage complex, mesh dense objects is still necessary.

With this said, if the option exists, preference should be towards modelling or sculpting high-resolution meshes for normal map production rather than using converted images, especially for those needing to reflect structure rather than surface detailing.