Topic: [MD5] EXPORT script for Blender 2.6x (OPEN)

[keless]

Here is the current WIP script:http://dl.dropbox.com/u/1556553/io_export_md5.9-15-2010.py

EDIT: for Blender 2.54 beta

* The joints section at the top of the file seems CORRECT.

* Actual vertex data seems CORRECT.

* Tex UV coordinates are WRONG.

* Bone weights are INCONCLUSIVE (since I don't export animation data to test them out yet).

This script will FAIL on line 500 on some models due to index OOB error, which I'm currently looking at.

[kat]

Couldn't get this to run for some reason. Tried loading it into a text view and Alt+P'd it but that doesn't run scripts at present. Also dropped it into scripts/io and had to change the name to "export_md5.py" for it to appear in the export list, but it shows up as a list heading rather than an active script for me.

This is with 2.53beta running on Vista.

[keless]

Couldn't get this to run for some reason. Tried loading it into a text view and Alt+P'd it but that doesn't run scripts at present. Also dropped it into scripts/io and had to change the name to "export_md5.py" for it to appear in the export list, but it shows up as a list heading rather than an active script for me.

This is with 2.53beta running on Vista.

Sorry, I forgot to specify, this is for the latest public beta version of Blender which is now 2.54  (they changed the module registration methods which broke 2.53 scripts, so it wont even run if you try it in 2.53)

Also; small update to the script, which fixed the index OOB error, however the model that had that error now compiles but vertices are messed up when rendered.

[kat]

Tried two tests with this once I got it loaded into Blender ... btw... "grrrrr" on the fact that they've changed things around internally yet again. Distributied scripts are in a different location, if you put customs in there they don't work, so thankfully it still seems that "scripts/io" is where they should be placed.

Tried the script with the source file version of bob (linked to above) loaded into 2.54 and got some output but just a few lines;

Code: [Select]

MD5Version 10
commandline "Exported from Blender by export_md5.py by Paul Zirkle"

numJoints 0
numMeshes 1

joints {
}

mesh {
shader "body"
numverts 0
numtris 0
numweights 0
}Produces the same output regardless as to whether the mesh has a parent or not. A second test on a simple textured cube has Blender go through the motions of exporting without actually producing a file; it's been a long time since I used MD5 meshes so I can't remember if there were some necessary requirements for export - vertex_groups and or Armatures etc?.

[EDIT]Console writes the following error on the simple cube

Code: [Select]

Exporting selected objects...
Processing mesh: Cube
Traceback (most recent call last):
  File "C:\Users\[user]\AppData\Roaming\Blender Foundation\Blender\2.54\scripts\io\
io_export_md5.py", line 637, in execute
    save_md5(settings)
  File "C:\Users\[user]\AppData\Roaming\Blender Foundation\Blender\2.54\scripts\io\
io_export_md5.py", line 491, in save_md5
    ), matrix))
NameError: global name 'matrix' is not defined

[keless]

MD5 requires an armature

the samples I'm working with have 2 meshes (one head and one body) and a 30 bone armature which uses vertex groups to bind the bones: so I think yes, both are required.

Then to export, I select the two meshes and the armature (since the script pulls from the list of highlighted objects). You should only have ONE armature in the selection (the script will use the first armature it finds and ignore any others), but will export any number of meshes and ignore any other object types. The meshes can be Tri or Quad, but the script will cut quads down into tris, and throw away any faces with overlapping (forget the term; degenerate?) vertices.

looks like line 492 is a bug, since that should be "w_matrix" not "matrix" -- I renamed the variable from the original to be a little less generic (that is the obj.matrix_world matrix). Don't know why it not complaining about it in my tests, I guess its just accepting a null?

EDIT: just ran my samples thru der_ton's script on blender 2.49b and it seems like a lot of differences (not just a texUV Y thats flipped, but different bone coords and also more verts, tris and weights showing up!)


First thing I'm trying to address is the skeleton code, specifically the def treat_bone() function, since that should be the easiest.

Currently the position of the bones is wrong, but the rotation is correct.

In the original code, there is both a mat and pmat that is calculated, however pmat is stored in the Bone object but never used-- so I'll ignore it, leaving me with the following original code:

Code: [Select]

matrix = obj.getMatrix('worldspace')

def treat_bone(b, parent = None):
  ...
  mat = Blender.Mathutils.Matrix(b.matrix['ARMATURESPACE'])*matrix
  ...
  bone = Bone( ..., mat, ... )
  ...

Initially, I have translated this to Blender 2.5x as:

Code: [Select]

w_matrix = obj.matrix_world

def treat_bone(b, parent = None):
  ...
  mat = b.matrix_local * w_matrix
  ...
  bone = Bone( ..., mat, ... )
  ...

And the results of the first two bones in each case:

Code: [Select]

"Torso" -1 ( 0.019181 0.011407 7.975472 ) ( -0.707107 -0.000000 -0.000000 )       //2.49
"L_Up_Arm" 0 ( 0.019181 0.011407 12.575473 ) ( 0.000000 0.017672 0.707106 )           //2.49
"Torso" -1 ( 0.019181 -7.975471 0.011409 ) ( -0.707107 -0.000000 -0.000000 ) //2.54
"L_Up_Arm" 0 ( -0.187871 0.180156 12.571256 ) ( 0.000000 0.017672 0.707106 )            //2.54

In both cases, the rotations (the second set of xyz data) are correct, however the positions are off. Specifically the position of the root bone looks like "x -z y", while the second bone is much further deformed (probably error propagation). In which case the problem seems to be a coordinate system change in Blender from 2.4 to 2.5? Amirite?

[keless]

Okay this is weird as hell. I'm comparing the actual matrix values found in 2.4 and 2.5

The actual matrix values before the multiplication match up. Specifically:

obj.getMatrix('worldspace')  == obj.matrix_world ==
[1.000000, 0.000000, 0.000000, 0.000000]
[0.000000, 1.000000, 0.000000, 0.000000]
[0.000000, 0.000000, 1.000000, 0.000000]
[0.019181, 0.011407, 7.975471, 1.000000]

and

b.matrix['ARMATURESPACE'] == b.matrix_local ==
[1.000000, 0.000000, -0.000000, 0.000000]
[-0.000000, 0.000000, 1.000000, 0.000000]
[0.000000, -1.000000, 0.000000, 0.000000]
[0.000000, 0.000000, 0.000002, 1.000000]

however, it seems like the matrix multiplication result is DIFFERENT?!

in Blender 2.49b the result is:
[1.000000, 0.000000, -0.000000, 0.000000]
[-0.000000, 0.000000, 1.000000, 0.000000]
[0.000000, -1.000000, 0.000000, 0.000000]
[0.019181, 0.011407, 7.975472, 1.000000]

in Blender 2.54 the result is:
[1.000000, 0.000000, -0.00000, 0.00000]
[-0.00000, 0.000000, 1.000000, 0.00000]
[0.000000, -1.00000, 0.000000, 0.00000]
[0.019181, -7.975471, 0.011409, 1.0000]

ARRRGGG!! order of multiplication changed!

instead of multiplying the bone space * world matrix in 2.4, we need to multiply the world matrix * bone space in 2.5

updated script in place (http://dl.dropbox.com/u/1556553/io_export_md5.9-15-2010.py )

now I need to figure out why there is a major difference in the number of verts/tris/weights section. Probably a problem with the whole face_vertices code, and how it does or doesnt cut quads up into tris and handles cloning vertices (since I didn't fully understand the code when I was translating it to 2.5)

[kat]

If I'm understanding what you're saying there would it make any difference to *require* meshes be properly prepped *before* export so the script isn't doing what is effectively unnecessary work?

[EDIT]Good job so far. Results of export in the MD5 viewer. Mesh seems to be solid and everything in place, only UVW's are a messed up as per your comments above about that.

[keless]

If I'm understanding what you're saying there would it make any difference to *require* meshes be properly prepped *before* export so the script isn't doing what is effectively unnecessary work?

[EDIT]Good job so far. Results of export in the MD5 viewer. Mesh seems to be solid and everything in place, only UVW's are a messed up as per your comments above about that.

The md3 exporter for 2.5x _requires_ that the mesh already be 'triangulated' (CTRL+T) before its run. Personally, I'd rather not have this requirement, but it may be possible to copy the mesh data and run the Quad->Triangle algorithm on the copy and cull the degenerates before parsing into Face() s

I dont think the UV code is technically the problem right now, since flipping the Y/V coord worked. I think the texture problems you see are related to the improper way I'm counting up the faces/tris, and should work itself out when thats done. I know you dont see vertex problems with the guard char, but I dont see vertex problems with my frog character, and I do see plenty with my panda character.

[keless]

I've nailed down another problem in the script, specific to the getVertexInfluences code I had hacked out earlier.

In this example, the left bone name is what my function returns (always the same as the last bone name) and the right is what a hand-crafted lookup of the bone name returns.

influences for vert 1115 are:
R_Hip vs: L_Hip
R_Hip vs: L_U_Leg
R_Hip vs: R_Hip
influences for vert 1885 are:
R_Hip vs: Torso
R_Hip vs: L_Hip
R_Hip vs: R_Hip

This would probably account for the difference in WEIGHT values since there are less unique bones associated with each vertex as a result. We'll see how much this helps when I fix it.

EDIT: I've now fixed this issue, the bone influences are now returned correctly.

However, this does NOT increase the number of entries in the VERTS and WEIGHTS section. In my frog character the result is not graphically noticable. In my panda character, the body looks less like a mess of spaghetti and slightly more like a stick figure, but still not a panda.

I'm now looking into why exactly my number of VERTS/WEIGHTS doesnt match the original.

The latest: ( http://dl.dropbox.com/u/1556553/io_export_md5.9-16-2010.py )

EDIT2:  (sorry for all the updates, but it helps me think about it clearly to write out what I know)
* the number of VERTS in the Body sub-mesh of frogger.md5mesh generated from the 2.4 script is 2298
* the number of VERTS in the Body sub-mesh of frogger.md5mesh generated from my 2.5 script is 2010
* the number of vertices in the Body mesh of frogger in memory in Blender 2.4 AND Blender 2.5 is 2010
* weights are generated as a direct result of the list of verts
>> conclusion: the old mesh generates 288 extra verts that my script does not
* there are 3 places VERTS are generated; (1) once for each vertex in the blender mesh, (2) once AGAIN for each vertex shared by non-smooth faces because they have diff normals, (3) once AGAIN for each vertex with two separate UV coords for different faces

... carp, the original script has a weirdly tabbed for-if-else section that I 'fixed' the indentation of.. this might have broken something

[keless]

Progress: the frogger character is now 100% correct, but the panda character is still messed up.

(I made these models myself, and I'm a programmer, and there is no lighting in this shader, so dont worry about how crappy they are   )

Upon exporting the panda from the 2.4x script and comparing the results, I realized that the left/right hip bones aren't actually attached to the intended root bone (so there were effectively 3 root bones). However, after fixing this I still get the exact same result as the previously posted screenshot.

[kat]

Personally I'm glad to see you writing out what you are, so few people do this so it actually makes what they're doing more difficult to follow. So... keep doing it if you have a mind to

On the extra verts... so if the script reads that a vert is on the border of both a smoothing and UVW split it introducing extra vertices per each dataset? Meaning you end up with three vertices at any given junction?

With the panda character... how old is it and what version of Blender was used to originally make it? And has it been through any iterations in different versions of Blender up to and including 2.5?

I say this because that looks like a Blender 'data' problem that happens occasionally when working with old files that have been passed through and saved in a number of different versions of Blender, this is **especially** so with Armatures. To test the output as an MD5 it might be worth stripping the current rig and replacing it with a simple two bone set up, that way you can see if there's an issue with it or not on re-export. Cute characters btw

Not tested the new script yet but will post when done so, I've got a couple of rigged characters including the old version of Bob I can cross-test this with.

[keless]

re:verts -- I think technically you could end up with N verts per junction, where N = number of unique UV + normal pairs associated with that vertex. Average case will probably result in 1-n_faces verts per junction depending on whether or not the normals are same, and more on UV texture seams. In short: yes.

re:panda -- I think it was modeled in 2.48 and animated in 2.50, so it's probably worth remaking the skeleton.

meanwhile, I think I'm ready to move on to exporting md5anims next.

EDIT: Replaced the armature and the panda looks good now. Thanks for the tip, Kat; I might have been chasing my tail in the script code forever without it.

[keless]

Okay so I was never 100% clear on this in blender, and I want to make sure I have my head on straight about animations before interpreting their API.

When an artist is finished rigging, they select the armature and go into "Pose Mode". There they can modify bone positions and insert key-frame data (as a combination of Loc/Rot/Scale) per-bone at given time frames.

When this is done, it seems to automatically create an "Animation" child of that Armature (visible in the Outliner window), with an "Action" default named 'ArmatureAction'.

* I can delete or rename the action in the DopeSheet view (but no where else?)
* I cant seem to add other 'actions'
* I cant seem to add other 'animations'

* clicking the "snowflake" icon in the red strip found in the NLA Editor causes Bad Things to happen (seems to bake the action into an NLA track object which cant be edited? is this actually a Good Thing?)
* after creating a few of these, the animations seem to be floating around in memory not really attached to the 'armature' or the 'animation', however selecting one in the DopeSheet drop down and then playing it a few times seems to cause that 'action' to replace the single 'action' child of the 'Animation'
* I can create new NLA tracks, and have them consume the currently active 'action' in the NLA Editor, such that an 'Animation' object can have one "active" child 'action' and a number of NLA tracks each with their own 'action' (that might or might not also be the active 'action')

* in python, one can access the 'Animation' object from bpy.context.object.animation_data while an armature is selected in object mode (cant currently select it from bpy.data.armatures['armature name'].animation_data for some reason)

* from the animation_data, one can access a singular  .action object, or a list of .nla_tracks which seems to correspond to the results above.

THE QUESTION:

How does an artist that knows what they're doing set up character animations? To they simply have one long 'action' with all animations inserted in a row (keyframes 1-99 are 'idle', 100 - 199 are 'walk', 200 - 299 are 'kick') or do they somehow set up a series of separate NLA tracks each with their own 'action' child, meaning that the single 'action' child of the 'animation' object is just the 'currently selected action'

I need to know this so that I pull the animation(s) from the right place, instead of forcing animators to mess up their .blend to squeeze out .md5anim file(s)

[kat]

The engine you're exporting to or working with determines which animation process you use. There are basically two types or ways to animate.

• 1) individual animations
• 2) included or grouped animations

1) Most games work this way, meaning that you create separate animation tracks - 'run', 'walk' and so on - each of which is then exported out as a separate animation file, in this case you could have ten separate *.md5anim files and one *.md5mesh file.
2) Tech like Unity3D use this approach whereby all the animations are included into a single track which is exported out. In the engine you then set up the animations doing what you said - 1-99 = 'walk', 100-199 = 'run' and so on

Now with regards to the 'structure' they've changed that again. In previous versions you'd have the 'Armature' at the top and then the animations as a child connections;

• Armature
• >> animation 1
• >> animation 2
• >> animation 3

But they appear to have changed that so in the OOPS/Outliner view its now listing animations only on being actively selected like this;

• Armature
• >> animation 1

And again if a different action/animation is active;

• Armature
• >> animation 2

That's bound to change the internal data structure but at the same time it does mean that Blender is dealing with only the data that selected/active?

You can change the name of animations in the OOPS/Outline be Ctrl+LMB clicking the name to active 'edit' mode on that, you can only change user generated data though and not 'core' data - "Animation" as a named datablock can't be changed.

For the other stuff you mentioned, the "Action" editor is now a sub-editor of the main "Dopesheet" editor, so you have to go into Dopesheet and select "Action" from the dropdown menu in the header. Once there you can do all the usual edit/delete/new functions previously available.

[keless]

Okay, I'm starting to see how I can go in to the NLA Editor and select an animation (or add a new one) in the "animation data" section of the properties panel (hotkey 'N') of that window.

1) you dont mention NLA Tracks at all; useful? ignore them?
2) if not NLA tracks, then actions are basically floating in memory and only the "active" one is really visible at any one time, so I wouldn't be able to export more than one action at a time it seems?

Aside: Blender seems to have a couple of places (this, and texture image links come to mind) where stuff can be created and just floats around in memory unless its also attached to something, which bugs the carp out of me. Stupid myTexture.001, myTexture.002 IDs that I don't know exist and cant get rid off once I do. In my mind, EVERYTHING data wise should be accessible from the Outliner tree.