Technical Animation - Final Project: Spatial Keyframing in Unity (C#)
Updated: Jun 15
1. Problem Statement / Goals
The goal of this project is to create a toned down version of the 2005 paper, Igarashi’s Spatial Key framing for Performance-driven Animation. This paper introduces spatial key framing, a technique for performance-driven character animation. This is a technique which created animations from scratch and does not need any motion capture data for implementation.
Spatial Key frames for the system are defined as a cursor position associated with that spatial keyframe and also the associated pose that is a 3*3 matrix.
This paper uses Radial Basis Function for interpolation. For interpolation between two points each entry of the rotation matrix is interpolated using a radial basis function and the resulting matrix is then orthonormalized.
The Radial Basis Function is defined as follows it is very useful for scattered data interpolation, the basis function for this system as directed by the paper is the |x| function.
For each entry in 3* 3 matrix solve for dj such that f(x) = given pose at Cx,Cy,Cz (marker locations)
Cj, - Marker positions.
P(X) – degree one polynomial.
F(x) - radial function ( |X| in the paper)
For root part – translation motion
Calculate F(X) for the translation vector using this interpolation
Interpolation system relies on the assumption that the interpolation function f(x) can be found through a linear combination of basis functions. So our system needs to satisfy the equation f(xi) = hi, this equation gives rise to the following linear system, first we need to find the weights d1……dn and the linear term, these values will give us the interpolation system.
We obtain the interpolation function f(x) by solving the above linear system. We need to solve the linear system for each entry of the 3×3 rotation matrix. So we will get 9 interpolation function in total which will give us the transformation for our joint which will be included in a 3x3 interpolation matrix.
This matrix that we get needs to be orthonormalized as it is not a orthonormal rotation matrix, the method for orthonormalization outlined in the paper is that of iterative refinement.
This orthonormalized matrix is then applied to the concerned joint.
For this project I used Unity 3D and a 3D human model which was rigged using Maya’s Auto Rig Feature.
Step 1: Importing a fully rigged 3D model into Unity.
I imported this model in Unity 3D, and used a spherical collider where the figure joints were with which I was able to rotate the joints of the model during runtime. This model was modeled by me on an earlier semester.
Step 2: Creating Spatial Key frames.
In the final deliverable of the paper there is a mechanism for creating spatial key frames at runtime, I tried to replicate this functionality by having a translatable cube as my cursor, and this cursor sets a spatial key frame (sphere) when the set spatial key frame button is clicked.
It saves the cursor position and the pose to a text file SpatialKeyFrame.txt which has the following structure.
[0.03306098, 0.6888634, 0.7193463, 0.08315936]
Cursor[1.083972, 0.7394111, -9.309]
[0.5796403, 0.2554487, 0.3599544, -0.6849789]
Cursor[0.241819, 0.7230731, -9.309]
I am saving a quaternion here but converting it into a rotation matrix using Unity inbuilt Quaternion Library.
Step 3: Interpolating between spatial key frames.
The next step was interpolating between spatial key frames I tried out the Algorithm for 1D first, this algorithm worked fine after comparing with the solved examples.
As mentioned in the paper the procedure for the 3D rotations is very similar to the above 1D Examples, only difference is that using the 3*3 rotation matrices instead of giving the angles around the axis. The procedure followed is as mentioned in the paper itself except the orthonormalization process for which Unity.Vector3.orthonormalize is used.
4. Problems faced
· The biggest problem I faced was in understanding the algorithm for and how it works with respect to a 3*3 matrix, and what would be a hi for a 3*3 matrix. If I had more time I would have liked to understand how the coefficient matrix is formed, and what is the understanding behind the radial basis function.
· As it is mentioned in the paper when the spatial key frames are less than 4 and they are linearly or two dimensionally distributed. There are inconsistencies with respect to the interpolation mostly concerning scale. This could be due to the code itself or due to using inbuilt functions to convert quaternions to rotation matrices.
Unity Documentation for Orthonormalization: https://docs.unity3d.com/ScriptReference/Vector3.OrthoNormalize.html
MathNet for linear algebra operations: https://numerics.mathdotnet.com/
System.Numerics.Quaternion for Quaternion to Rotation Matrix conversion.
Spatial Keyframing for Performance-driven Animation T. Igarashi 1,3, T. Moscovich 2, and J. F. Hughes 2 https://www-ui.is.s.u-tokyo.ac.jp/~takeo/papers/squirrel.pdf
Modelling with Implicit Surfaces that Interpolate GREG TURK Georgia Institute of Technology JAMES F. O’BRIEN University of California, Berkeleyhttp://graphics.berkeley.edu/papers/Turk-MIS-2002-10/Turk-MIS-2002-10.pdf