## Overview

The project is aimed at designing and implementing a prototypical spline interface for skinning weights editing, which is inspired by the paper Spline Interface for Weights Editing and more intuitive and easier to use than common paint-based interfaces.

Specifically, the interface will be implemented in Maya using Maya plug-in APIs.

Duration: 2 weeks

Role: Programmer

Team: Solo

Platform: Maya, C++

## Techniques

The interface has three main parts.

The first part retrieves data from the original mesh and joints, namely the mesh verticies (MVs) and skinning weights. The second part allows users to create splines and specify the spline values. The third part uses original skinning weights, user defined splines and spline values as input and generate new weight values and set them back to the mesh and joints.

### Retrieve original data

As metioned before, original data of the mesh and joints are needed by the later steps.

First, the current interface requires MVs for users to create splines. On the one hand, the interface should provide a way to select some vertices as control vertices (CVs). On the other hand, the interface has to know the actual position of each MV for B-spline generation. Maya has public class MSelectionList for making selection and MItMeshVertex for interating among the selected MVs.

// Global selection list
MSelectionList selectionList;
MGlobal::getActiveSelectionList(selectionList);

// DAG path and component
MDagPath dagPath;
MObject component;

// DAG path and component of selected mesh
selectionList.getDagPath(0, dagPath, component);

// Mesh vertex iterator
MItMeshVertex vertexIt(dagPath, component);

Second, the interface retrieves and stores the original skinning weights for the future use when calculating new weights. Similarly, interface should provide a way for selection of the specific mesh and joints before getting the actual weights. Maya uses public class MFnSkinCluster to retrieve skinning related data.

// Global selection list
MSelectionList selectionList;
MGlobal::getActiveSelectionList(selectionList);

// DAG paths and components of selected joint and mesh
MDagPath jointDagPath, meshDagPath;
MObject jointComponent, meshComponent;

selectionList.getDagPath(0, jointDagPath, jointComponent);
selectionList.getDagPath(1, meshDagPath, meshComponent);

// ...

// Skin cluster
MItDependencyNodes iter(MFn::kSkinClusterFilter);
MFnSkinCluster skinCluster(iter.thisNode());

### Create splines

The original paper uses the standard cubic B-spline model, which is widely used for its smoothness and local controllability. In Euclidean space, a B-spline curve is defined as a weighted sum of B-spline basis functions. Given a set of anchor points, a B-spline curve is given by:

$s(u)=\sum_{i=0}^{n-1}c_i^pR_{i,r}(u)$

There are plenty ways to create B-splines since it’s already a very common used model. Maya has its own APIs for creating B-splines as well, wrapped in the public class MFnNurbsCurve.

The only thing that should be concerned about is the form of the spline and how to specify the required parameters. The spline should be working correctly as long as it’s closed, however, to make the visualization better, we applied a more restrictive form, a periodic form.

Maya’s non-uniform rational B-spline (NURBS) curve is defined by an array of CVs, an array of knot values, a degree and a form, which is already decided to be a periodic form.

// Spline constants
const unsigned deg = 3;
const unsigned nspans = vertexIt.count() / splineNumber;
const unsigned ncvs = nspans + deg;
const unsigned nknots = nspans + 2 * deg - 1;

After defining and calculating all the required parameters, the spline can be easily created by calling the create method:

// Control vertices and knots used to create splines
MPointArray controlVertices;
MDoubleArray knotSequences;

// Create periodic spline
MFnNurbsCurve curve;
curve.create(controlVertices, knotSequences, deg,
MFnNurbsCurve::kPeriodic, false, false, MObject::kNullObj, &status);

The spline values are relatively easy to get since they are purely an array of floating numbers. Users can simply specify them in the command.

### Calculate weights

With splines and spline values specified and original weights retrieved and stored, we can calculate the new weights and set them back to the mesh and joints. The original paper has introduced several methods of weight conputation, including interpolation-based method, diffusion flow method and hybrid method. In this project, interpolation-based method with Laplacian operator is applied.

According to the paper, the problem can be formulated within a surface with the linear equality constraint of splines:

$w^s=\arg\min_x x^TQx\ \text{ subject to }Ax=s^i$

After the simplification, the mesh is a simple polygon cylinder, the splines can be considered as a set of MVs, and the original weights can be considered as simply a 2D array of floating numbers. What’s more, the MVs and weights can be easily associated with indices. Therefore, the weight interpolation can be simplified as a 2D image filtering problem, which can be solved by Laplacian interpolation. The book Numerical Recipes 3rd Edition: The Art of Scientific Computing has given a very detailed explanation and open-sourced implementation.

MatDoub matrix = NRmatrix<double>(49, 40, weights);
Laplace_interp interp(matrix);

interp.solve();
newJointWeights.clear();
for (int i = 0; i < interp.mat.nrows(); i++) {
for (int j = 0; j < interp.mat.ncols(); j++) {
newJointWeights.append(interp.mat[i][j]);
}
}

## Follow-up

### More complicated models and meshes

Maya has its own rules of interating among all the MVs. The interface now only supports polygon cylinders or meshes that are deformed from polygon cylinders, because the current algorithm hasn’t considered the order of the MVs. It now doesn’t affect the weight calculation part so the weight values can be computed correctly. But the splines are not correctly generated.

Therefore, to support more complicated models and meshes, the actual world position of each MV should be considered so as to know the order of the MVs when generating splines.

### More delicate user interface

One of the reasons why the original paper is very impressive is because the intuitive interface and the user experience can save users a lot of time compared to paint-based interfaces. The interface in this project is mainly focusing on implemeting the algorithm process, so the interface can hardly be considered effective and intuitive.

Therefore, to actually help artists and animators reduce their work, more delicate and intuitove user interface should be created.

### Spline editing

Similar to friendly interface, spline editing makes no difference in terms of the algorithm itself, but it is one of the factors that make the interface very easy to use and can be possibly used in real production.

### Spline on mesh surface

In this project, the splines are generated using MVs as CVs, so they are in fact inside the mesh, which has to be changed when adding spline editing to the interface. The original paper uses Phong Projection to project the spline to the surface, which is a very important step.

### Weight computation

As mentioned before, the original has applied multiple weight computation methods for better performance and effect. For example, interpolation-based methods are good at distributing values for the area between two iso-values, while the diffusion flow method is a better choice to calculatng weights for the rest of the area. Besides, even just for interpolation-based methods, different operators or energy can have different performance and results.

Therefore, to improve the interface, a lot of different methods should be applied, compared and optimized, if necessary.

### Weight freezing

Mostly, the skinning weights will be edited and modified for several times. So one thing the interface should do is to maintain the weights of the current joint when modifying those of the next joint, which is also addressed by the original paper. For the current interface, it should be easy to do by simply storing all the modified weights and setting them back the mesh when all edits are done.

## References

### Original Paper

Go to the original paper Spline Interface for Weights Editing by SEUNGBAE BANG and SUNG-HEE LEE, Korea Advanced Institute of Science and Technology via this link.

### Math library

Maya provides a reference for detailed documentation of the classes, methods, types and constants provided by the Maya API along with cross-linked source for the code samples provided in the Maya Development Kit.

The enterprise of the book provides a very profession webpage for all used resources in the book, including the book iteslf and the code implemetations.

### Maya Plug-in Tutorial

Autodesk has a set of tutorial pages for writing plug-ins, which provides detailed documentations and examples. Here is a link to the tutorial page of Maya 2020.