What are smooth groups and mesh smoothing on models for?

There are generally two ways a 3D model will have 'smoothing' applied to it - 1) information is applied to an object and exported from a 3D application as part of the file. 2) a global setting applied to the model in some way by a game engine. The following tutorial will look at an example of this by investigating how a game like Doom 3 handles models and smoothing information.

Smoothing and model formats ^

There are 3 model formats that Doom 3 can use by default.

• *.ASE : ascii text based format native to 3DStudio Max.

• *.LWO : binary 'locked' format native to NewTeks LightWave.

• *.MD5mesh : proprietary format developed for Doom 3 by id.

As mentioned briefly above, each of these formats use smooth-groups and mesh smoothing in one of two ways, either;

• They use smoothgroup information present in the mesh object itself, usually assigned as the model is being edited in a 3D application - typically via 'hard' or 'soft' edges, vertex/face 'splits', vertex/face 'groups', 'detached' (sub)meshes and so on.

Or;

• The information present in the mesh pertaining to smoothing is completely ignored, the engine technology applies its own either as a shader/material value, or as a general global value applied to at the object level (smoothing applied over the entire mesh).

ASE and LWO tend to do the former, i.e. uses smoothgroup information present in the mesh. Whereas MD5mesh tends to do the latter, ignores the smoothgroup info in the mesh and applies it's own a 'global' settings.

The game engine, models and smoothgroups ^

Essentially yes, but the way you physically apply them to a mesh may need to change slightly, although Doom 3 does and can read and use smoothgroup info it's actual implementation seems a bit flaky in game to say the least, many modelers have found that applying smoothgroups in the normal manner inconsistent and unreliable often causing bad lighting artifacts in game because the system can't figure out what's going on with the model.

Creating smoothgroups on game models ^

The only guaranteed way to control smoothgroups is to 'force' them, by utilising the smoothgroups 'zones' normally set up when building a model in your 3D app. What this means is that you have to create physical divisions along the edges of your models wherever you have a meeting of any given number of smoothgroup zones.

The way to do this is to split the mesh up into sub-objects - these aren't separate mesh objects as they're still physically connected to the parent object. Think of a box. The box itself is the 'parent object'; it's what the object 'is'. Now look at the sides, they give the box it's physical attributes; the 'how it's made' in effect, each of these sides can be considered as a 'sub' or separate components in their own right; left, top, bottom, right, front and back - these are the 'sub-objects'.

Take a look at the following images, although self explanatory (and done in Blender 3D) what you're seeing is one side of the object being split away into a sub-object to create a clean smoothgroup between it and the box, remember it's still part of the parent object, the box.

Image 1 show the box with a global smoothgroup, notice the the harder shadows on the bottom edge, that would get over exaggerated in Doom 3 and cause a shadow artifact; the game won't be able to tell what's supposed to happen there from the point of view of how the lighting is applied to the box.

Image 2 below shows a the box in edit mode with a face selected ready for splitting away. The basic principle is the same for all 3D applications so all you need do is select the polygons you need based on each smoothgroup grouping and then detach or split that group away from the other polygons. Don't separate the smoothgroup into a completely separate objects, they need to still be attached to the parent object.

Design note : having said that though, you may find it easier to completely separate the selected polygons into their own objects (so they can only be selected in a non 'edit' mode), then once done on the whole mesh simply select all the separate objects and 'attach' (to use a 3DS Max term) or connected them all back into one single object. Because this usually doesn't weld the vertices back together by default you'll be left with clean edges along the boundaries of what were separate objects before their being joined together.

Images 3 and 4 show the 'paired' vertices on one corner of the box that are created as a result of the separation of the face. This is how splitting the mesh and smoothgroups work, vertices along the edge of a selection are 'duplicated' so each section has it's own vertex edge.

The final image below shows the box with 2 smoothgroups; the face with it's own smoothgroup and the rest of the box with another. You can now clearly see the edge that gets created.

Obviously you'll get different result depending on what the object is but in essence you need to split the mesh based on the smoothgroup 'zones' initially created; for the box in this example each face could be separated out into it's own sub-object because the model might require hard edges all round. Speaking of hard edges, if the model require 'softer' edges you may need to use additional polygons to 'round off' or 'bevel' an edge. If that bevel is then made part of another smoothgroup it will create a nice simple 'curved' edge.

Conclusion ^

Although you may find for the most part that ASE and LWO files are 'OK' and give you the result you want, MD5meshes won't, if you want a smoothgroup on those you need to use the method above. Be mindful however of discrepancies when animating models for MD5mesh output as splitting faces like this and creating extra vertices effects the weight painting and parenting to the bones of a skeleton, it can result in vertices occupying the same co-ordinates in space moving very differently relative to the bones of an armature and each other.