Click here to Skip to main content
15,867,141 members
Articles / Multimedia / DirectX

Loading and Rendering Milkshape 3D Models with Animation and Skinning

Rate me:
Please Sign up or sign in to vote.
4.81/5 (13 votes)
18 May 2011CPOL5 min read 88K   4.6K   31   24
This article shows how to load Milkshape ms3d binary files, animate and display them with OpenGL

Introduction

Skeletal animation is a technique used in 3D rendering which uses an exterior shell composed of vertices and an internal skeleton for the animation. To avoid breaking the model, each vertex is assigned a set of bones and weights used to compute its final position based on the animation key frames.

In modern application, this technique is used frequently. Autodesk Media and Entertainment is one of the companies that produces software used in films, commercials and computer games. Among its current products, we can find Maya and 3DStudio MAX. The max file format is not very well documented and the 3Ds is just there for legacy purposes. In some cases, if you export and re-import from 3DSMax, the animation and models simply break. Autodesk offers developers SDK support for implementing their own exporters and importers but this takes a lot of time. On the other hand, MS3D is a complete documented file format with source code examples, in Milkshape 3D is a very rapid and easy to use application, making it easy to load and render their native file format. So if you want 3D modeling for your application, you can simply turn to Milkshape for creating and animating your objects.

The code is written in JAVA, but I also have a C# version with XNA or DirectX Managed. If you want the code, just let me know and I’ll email it or maybe just make it public.

milkshape_models.png

Further reading on skeletal animation and skinning can be done at http://en.wikipedia.org/wiki/Skeletal_animation. I recommend that you familiarize yourself more with this technique before you get started on the code.

Topics Covered by this Article

  • JOGL (java opengl)
  • Skeletal animation
  • Milkshape 3D model loading and rendering
  • GLSL
  • Hardware skinning

Specifications

The application includes the following features:

  • Vertex manipulation in GLSL
  • Loading and rendering Milkshape files
  • Skeletal animation
  • OpenGl rendering

Technical Information

  • Code written in JAVA Netbeans
  • Uses JOGL for rendering
  • Uses GLSL for hardware skinning

Requirements

  • Netbeans
  • OpenGL enabled video card with GLSL support (tested on Nvidia, hardware skinning fails on ATI and older Nvidia chipsets)
  • Milkshape 3D
  • JOGL

Using the Application

A demo of the application can be found at http://inline.no-ip.org/testing/java/ViewApplet2 (software skinning version) or http://inline.no-ip.org/testing/java/ViewApplet3 (hardware skinning version).

Using the Code

The provided code is meant to work in Java enabled browsers as a Java applet. The skeletal animation class is called MilkshapeModel and the constructor receives an absolute URL from which to download the required files. Textures have to be in the same directory as the model. The Boolean parameter represents the hardware skinning flag. If it’s set to true, it tries to download and compile the GLSL VertexShader for hardware skinning.

C#
model = new MilkshapeModel(new URL
	("http://localhost:8080/WebApplication/dwarf2.ms3d"), drawable.getGL(), true);

The Basic Milkshape 3D File Format is:

  • Header
    • "MS3D000000" followed by version number (ver. 3 or 4)
  • Vertex Data
    • Coordinates for the vertices
  • Triangle Data
    • Pointers to vertices, as well as surface normals
  • Group Data (object/mesh)
    • Group name and pointers to triangles
  • Material Data
    • Color details
  • Bone data
    • Animation data

The supported file format is version 4. The model loader is based on the C++ binary milkshape loader example provided by http://chumbalum.swissquake.ch/.

Skeletal Animation

In this example, skeletal animation takes place in two steps. The bone setup in which the key frame interpolation is computed and matrices are updated and the vertex update, where vertex positions are obtained using attached bones and weights. This is done either software or hardware by using a shader. Software takes a lot of computing power and is dependent on the number of vertices. Also if we add more models in the same scene, the slowdown is visible. Shaders execute much faster and use GPU power, so we can use the CPU for other operations.

In our case, we have a hierarchy of bones, each having its own set of key frames. These give rotation and translation data. The first step is to locate two key frames that contain the given frame and interpolate the result. Also we need to concatenate results with the parent joint. Joints are preordered in the array so now when we evaluate a Joint, we know we have already evaluated its parent.

C#
//bone evaluation
for (int i = 0; i < numJoints; i++) {
    EvaluateJoint(i, frame);
}
C#
private void EvaluateJoint(int index, float frame) {
    MilkshapeJoint joint = Joints[index];

    //
    // calculate joint animation matrix, this matrix will animate matLocalSkeleton
    //
    float[] pos = {0.0f, 0.0f, 0.0f};
    int numPositionKeys = (int) joint.positionKeys.length;
    if (numPositionKeys > 0) {
        int i1 = -1;
        int i2 = -1;

        // find the two keys, where "frame" is in between for the position channel
        for (int i = 0; i < (numPositionKeys - 1); i++) {
            if (frame >= joint.positionKeys[i].time &&
        frame < joint.positionKeys[i + 1].time) {
                i1 = i;
                i2 = i + 1;
                break;
            }
        }

        // if there are no such keys
        if (i1 == -1 || i2 == -1) {
            // either take the first
            if (frame < joint.positionKeys[0].time) {
                pos[0] = joint.positionKeys[0].key[0];
                pos[1] = joint.positionKeys[0].key[1];
                pos[2] = joint.positionKeys[0].key[2];
            } // or the last key
            else if (frame >= joint.positionKeys[numPositionKeys - 1].time) {
                pos[0] = joint.positionKeys[numPositionKeys - 1].key[0];
                pos[1] = joint.positionKeys[numPositionKeys - 1].key[1];
                pos[2] = joint.positionKeys[numPositionKeys - 1].key[2];
            }
        } // there are such keys, so interpolate using hermite interpolation
        else {
            MilkshapeKeyFrame p0 = joint.positionKeys[i1];
            MilkshapeKeyFrame p1 = joint.positionKeys[i2];
            MilkshapeTangent m0 = joint.tangents[i1];
            MilkshapeTangent m1 = joint.tangents[i2];

            // normalize the time between the keys into [0..1]
            float t = (frame - joint.positionKeys[i1].time) /
        (joint.positionKeys[i2].time - joint.positionKeys[i1].time);
            float t2 = t * t;
            float t3 = t2 * t;

            // calculate hermite basis
            float h1 = 2.0f * t3 - 3.0f * t2 + 1.0f;
            float h2 = -2.0f * t3 + 3.0f * t2;
            float h3 = t3 - 2.0f * t2 + t;
            float h4 = t3 - t2;

            // do hermite interpolation
            pos[0] = h1 * p0.key[0] + h3 * m0.tangentOut[0] +
        h2 * p1.key[0] + h4 * m1.tangentIn[0];
            pos[1] = h1 * p0.key[1] + h3 * m0.tangentOut[1] +
        h2 * p1.key[1] + h4 * m1.tangentIn[1];
            pos[2] = h1 * p0.key[2] + h3 * m0.tangentOut[2] +
        h2 * p1.key[2] + h4 * m1.tangentIn[2];
        }
    }

    float[] quat = {0.0f, 0.0f, 0.0f, 1.0f};
    int numRotationKeys = (int) joint.rotationKeys.length;
    if (numRotationKeys > 0) {
        int i1 = -1;
        int i2 = -1;

        // find the two keys, where "frame" is in between for the rotation channel
        for (int i = 0; i < (numRotationKeys - 1); i++) {
            if (frame >= joint.rotationKeys[i].time &&
        frame < joint.rotationKeys[i + 1].time) {
                i1 = i;
                i2 = i + 1;
                break;
            }
        }

        // if there are no such keys
        if (i1 == -1 || i2 == -1) {
            // either take the first key
            if (frame < joint.rotationKeys[0].time) {
                AngleQuaternion(joint.rotationKeys[0].key, quat);
            } // or the last key
            else if (frame >= joint.rotationKeys[numRotationKeys - 1].time) {
                AngleQuaternion(joint.rotationKeys[numRotationKeys - 1].key, quat);
            }
        } // there are such keys, so do the quaternion slerp interpolation
        else {
            float t = (frame - joint.rotationKeys[i1].time) /
    (joint.rotationKeys[i2].time - joint.rotationKeys[i1].time);
            float[] q1 = new float[4];
            AngleQuaternion(joint.rotationKeys[i1].key, q1);
            float[] q2 = new float[4];
            AngleQuaternion(joint.rotationKeys[i2].key, q2);
            QuaternionSlerp(q1, q2, t, quat);
        }
    }

    // make a matrix from pos/quat
    float matAnimate[][] = new float[3][4];
    QuaternionMatrix(quat, matAnimate);
    matAnimate[0][3] = pos[0];
    matAnimate[1][3] = pos[1];
    matAnimate[2][3] = pos[2];

    // animate the local joint matrix using: matLocal = matLocalSkeleton * matAnimate
    RecConcatTransforms(joint.matLocalSkeleton, matAnimate, joint.matLocal);

    // build up the hierarchy if joints
    // matGlobal = matGlobal(parent) * matLocal
    if (joint.parentIndex == -1) {
        //memcpy(joint.matGlobal, joint.matLocal, sizeof(joint.matGlobal));
        for (int k = 0; k < joint.matLocal.length; k++) {
            System.arraycopy(joint.matLocal[k], 0,
    joint.matGlobal[k], 0, joint.matLocal[k].length);
        }
    } else {
        MilkshapeJoint parentJoint = Joints[joint.parentIndex];

        RecConcatTransforms(parentJoint.matGlobal, joint.matLocal, joint.matGlobal);
    }
}
private void RecConcatTransforms(float in1[][], float in2[][], float out[][]) {
    out[0][0] = in1[0][0] * in2[0][0] + in1[0][1] *
        in2[1][0] + in1[0][2] * in2[2][0];
    out[0][1] = in1[0][0] * in2[0][1] + in1[0][1] * in2[1][1] + in1[0][2] * in2[2][1];
    out[0][2] = in1[0][0] * in2[0][2] + in1[0][1] * in2[1][2] + in1[0][2] * in2[2][2];
    out[0][3] = in1[0][0] * in2[0][3] + in1[0][1] *
        in2[1][3] + in1[0][2] * in2[2][3] + in1[0][3];
    out[1][0] = in1[1][0] * in2[0][0] + in1[1][1] * in2[1][0] + in1[1][2] * in2[2][0];
    out[1][1] = in1[1][0] * in2[0][1] + in1[1][1] * in2[1][1] + in1[1][2] * in2[2][1];
    out[1][2] = in1[1][0] * in2[0][2] + in1[1][1] * in2[1][2] + in1[1][2] * in2[2][2];
    out[1][3] = in1[1][0] * in2[0][3] + in1[1][1] *
        in2[1][3] + in1[1][2] * in2[2][3] + in1[1][3];
    out[2][0] = in1[2][0] * in2[0][0] + in1[2][1] * in2[1][0] + in1[2][2] * in2[2][0];
    out[2][1] = in1[2][0] * in2[0][1] + in1[2][1] * in2[1][1] + in1[2][2] * in2[2][1];
    out[2][2] = in1[2][0] * in2[0][2] + in1[2][1] * in2[1][2] + in1[2][2] * in2[2][2];
    out[2][3] = in1[2][0] * in2[0][3] + in1[2][1] *
        in2[1][3] + in1[2][2] * in2[2][3] + in1[2][3];
}

Hardware Skinning

After this, each vertex needs to be transformed according to the bones and weights. Milkshape uses a maximum of 4 bones per vertex, which is perfect for our vertex shader because we can store both bone ids and bone weights in two vec4 structures as vertex attributes.

GLSL Vertex Shader

C#
attribute vec3 normal;
attribute vec4 weight;
attribute vec4 index;
attribute float numBones;
uniform mat4 matGlobal[100];
uniform mat4 matGlobalSkeleton[100];
uniform vec4 color;
uniform vec4 lightPos;
float DotProduct(in vec4 x, in vec4 y) {    
return x[0] * y[0] + x[1] * y[1] + x[2] * y[2];
}
void VectorRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]);
    rez[1] = DotProduct(in1, in2[1]);
    rez[2] = DotProduct(in1, in2[2]);
}
void VectorIRotate(in vec4 in1, in mat4 in2, out vec4 rez) {
    rez[0] = in1[0] * in2[0][0] + in1[1] * in2[1][0] + in1[2] * in2[2][0];
    rez[1] = in1[0] * in2[0][1] + in1[1] * in2[1][1] + in1[2] * in2[2][1];
    rez[2] = in1[0] * in2[0][2] + in1[1] * in2[1][2] + in1[2] * in2[2][2];
}
void VectorTransform(in vec4 in1,in mat4 in2, out vec4 rez) {
    rez[0] = DotProduct(in1, in2[0]) + in2[0][3];
    rez[1] = DotProduct(in1, in2[1]) + in2[1][3];
    rez[2] = DotProduct(in1, in2[2]) + in2[2][3];
}
void VectorITransform(in vec4 in1, in mat4 in2, out vec4 rez) {
    vec4 tmp; 
    tmp[0] = in1[0] - in2[0][3];
    tmp[1] = in1[1] - in2[1][3];
    tmp[2] = in1[2] - in2[2][3];
    VectorIRotate(tmp, in2, rez);
}

void main(){
    vec4 transformedPosition = vec4(0.0);
    vec3 transformedNormal = vec3(0.0);
    vec4 curIndex = index;
    vec4 curWeight = weight;
    for (int i = 0; i < int(numBones); i++)
    {
        mat4 g44 = matGlobal[int(curIndex.x)];
        mat4 s44 = matGlobalSkeleton[int(curIndex.x)];
        vec4 vert = vec4(0);
        vec4 norm = vec4(0);
        vec4 tmpNorm = vec4(0);
        vec4 tmpVert = vec4(0);
        VectorITransform(gl_Vertex, s44, tmpVert);
        VectorTransform(tmpVert, g44, vert);//g
        vert[3]=1;
        transformedPosition += vert * curWeight.x;

        vec4 preNormal=vec4(normal.xyz,1);
        VectorIRotate(preNormal, s44, tmpNorm);
        VectorRotate(tmpNorm, g44, norm);
        norm[3]=1;
        transformedNormal += vec3(norm.xyz) * curWeight.x;

        // shiftam sa avem urmatoarele valori pentru pondere si indice de os
        curIndex = curIndex.yzwx;
        curWeight = curWeight.yzwx;
    }
    gl_Position = gl_ModelViewProjectionMatrix * transformedPosition;
    transformedNormal = normalize(transformedNormal); 
    gl_FrontColor = dot(transformedNormal, lightPos.xyz) * color;
} 

The code above is also implemented in software version for those who don’t have GLSL support.

For more reading on GL<code>SL, try http://www.opengl.org/registry/doc/GLSLangSpec.4.00.8.clean.pdf.

The GLSL is just for demonstration purposes. It doesn’t handle very well and wastes a lot of memory on the uniform matrices, failing on a lot of video cards because of this. However, if you want to test it, use a model with fewer bones and adjust the size of the matrices to suit your needs.
We animate the model using AdvanceAnimation:

C#
if (lastTick == 0) {
            lastTick = Calendar.getInstance().getTimeInMillis();
        }
        long currentTick = Calendar.getInstance().getTimeInMillis();
        long delta = currentTick - lastTick;
        lastTick = currentTick;
        //model.AnimationFPS=120;
        model.AdvanceAnimation((float) delta / 1000);

Conclusion

This article is an introduction to OpenGL software and hardware skinning using Java and Milkshape3D model loading. It’s not optimized, but you can use a lot of it in your applications. Hope this helps and if you make any good changes to the code, please let me in on it :).

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Research Institute For Artificial Intelligence, Ro
Romania Romania
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
PraiseExplorer Pin
www.pixelup.sk31-May-16 11:47
www.pixelup.sk31-May-16 11:47 
QuestionHelp Pin
Farouk2412-May-15 10:04
Farouk2412-May-15 10:04 
QuestionExcllent article! Pin
bitzhuwei18-Jul-13 21:28
bitzhuwei18-Jul-13 21:28 
QuestionDiablo? Pin
Reelix17-Aug-11 5:29
Reelix17-Aug-11 5:29 
AnswerRe: Diablo? Pin
Tiberiu Boros23-Nov-11 13:50
Tiberiu Boros23-Nov-11 13:50 
GeneralThis file can't be opened in Visual Studio 2010: "MilkshapeViewerXNA" => "The project type is not supported by this installation" Pin
RedDk26-May-11 13:22
RedDk26-May-11 13:22 
GeneralRe: This file can't be opened in Visual Studio 2010: "MilkshapeViewerXNA" => "The project type is not supported by this installation" Pin
Tiberiu Boros5-Jun-11 20:57
Tiberiu Boros5-Jun-11 20:57 
GeneralThank you for the update! Pin
LowPoly3D7-Feb-11 7:48
LowPoly3D7-Feb-11 7:48 
Questionit was excellent Pin
araelec29-Jan-11 22:54
araelec29-Jan-11 22:54 
GeneralVery nice Article Pin
KevinMO29-Jan-11 13:49
KevinMO29-Jan-11 13:49 
GeneralRe: Very nice Article Pin
Tiberiu Boros6-Feb-11 21:19
Tiberiu Boros6-Feb-11 21:19 
GeneralMy Vote of 5! Pin
Sharjith24-Jan-11 8:02
professionalSharjith24-Jan-11 8:02 
GeneralMilkshape 3d models Pin
rpraver24-Jan-11 4:05
rpraver24-Jan-11 4:05 
GeneralVery Nice! Pin
LowPoly3D21-Jan-11 22:46
LowPoly3D21-Jan-11 22:46 
GeneralRe: Very Nice! Pin
Tiberiu Boros22-Jan-11 12:02
Tiberiu Boros22-Jan-11 12:02 
GeneralRe: Very Nice! Pin
LowPoly3D6-Feb-11 12:44
LowPoly3D6-Feb-11 12:44 
GeneralRe: Very Nice! Pin
Tiberiu Boros6-Feb-11 21:13
Tiberiu Boros6-Feb-11 21:13 
GeneralMy vote of 5! Pin
ysliew77@yahoo.com21-Jan-11 3:28
ysliew77@yahoo.com21-Jan-11 3:28 
GeneralRe: My vote of 5! Pin
Tiberiu Boros21-Jan-11 4:06
Tiberiu Boros21-Jan-11 4:06 
GeneralRe: My vote of 5! Pin
Tiberiu Boros6-Feb-11 21:19
Tiberiu Boros6-Feb-11 21:19 
GeneralGood job Pin
Shane Story21-Jan-11 3:02
Shane Story21-Jan-11 3:02 
GeneralRe: Good job Pin
Tiberiu Boros21-Jan-11 3:04
Tiberiu Boros21-Jan-11 3:04 
GeneralMy vote of 5 Pin
Shane Story21-Jan-11 3:00
Shane Story21-Jan-11 3:00 
GeneralMy vote of 4 Pin
Slacker00721-Jan-11 0:31
professionalSlacker00721-Jan-11 0:31 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.