What's on the Agenda

I thought I'd do a quick post about what I'm working on now and planning for the next few weeks, as well as the remainder of the project.

• Thorough review of related work - I reviewed previous work when I first proposed the project and have found other relevant work as I've gone on. I'm going to through all the work again to determine the best direction to go in as I get into the major part of the project.


• Implement constraints - So far, I'm only generating overlays without constraints. The next step will be to implement constraints when automatically generating overlays, primarily physics checks for now.


• Analyzing results - After the constraints are implemented, I will thoroughly analyze the resulting overlays to see what works and what can be improved.


• Game environment - I'll create a game environment in OGRE to visualize the overlays through a game character.

I'm going to deviate from my original timeline a little bit. I don't think I'll spend nearly as much time on the game portion as I originally thought. Instead, the game environment will primarily be another means of viewing the overlays, and hopefully will make the results look more interesting. I'll be focusing much more of my time in analyzing the overlays and trying different methods to improve the results when automatically generating them.

Automatically Generating Overlays

The key part of this project is generating a large number of overlays and analyzing them. I implemented a feature in the user interface where you can generate overlays by specifying one folder of lower body motions and one folder of upper body motions.



I thought this would be a useful way to generate overlays especially when I implement constraints. This can easily be extended into a console application to generate a massive amount of different combinations.

We can quickly inspect the overlays once they are generated. I structured the output so that every upper body motion has a different folder, and the files within it are named lowerbodymotion_upperbodymotion.amc with their respective names.

Fixing Twitches

We noticed that there was some twitching in some of the overlays. After observing and playing around it with it a bit, I found that the problem lied in transplanting everything above the root. Update: Ignoring the lower back joint and transplanting everything above it looks better but loses some of the movement of the motion.



I'm also keeping in mind other methods of alignment, namely spatial alignment mentioned in the paper "Splicing Upper-Body Actions with Locomotion" by Heck, Kovar, and Gleicher. This involves finding a local rotation of the pelvis that best aligns the shoulders and spine of the upper body motion with that of the lower body motion. The best rotation is computed using point clouds.

Update: Something like the method mentioned above seems like a good way to start.

Interpolated Overlays

Last week, I implemented the ability to create interpolated overlays as well as specify the interpolation percentage. Here is a demo of the results:



As you can see, the lower body motion is a character jogging, and the upper body motion is a character standing still and drinking from a cup. In the basic transplant, one of the arms barely moves while the character is jogging. This is very noticeably unnatural. In the interpolated transplant, the arms move somewhat believably but the drinking motion is no longer apparent, and the overall result looks strange.

Perhaps a better solution would be to only transplant certain limbs rather than the entire upper body? There are plenty of things to think about in terms of improving the results of these overlays.

For this week, I'll be implementing the ability to automatically generate overlays.

Finding More Motions

Since this project relies on a useful set of lower body motions and upper body motions, it is important to first find some good motions we can use.

I've been browsing through the CMU mocap database (http://mocap.cs.cmu.edu). All we have to do is downsample a relevant motion and we can use it in our own framework.

For now, I've found a couple motions of a character standing still and drinking something that will be very useful in creating examples. Throughout the project, I'll try to incorporate some new motions that we can use to augment our results.

Creating Basic Overlays

I've implemented a tool for creating basic overlays. You can load any two motions (AMC files) and the upper body limbs of the second motion will be transplanted to the lower body limbs of the first motion. The root of the body from the second motion is first aligned with the body from the first motion (in position and orientation) before the transplanting. These are just basic transplants for now so it is expected that motions will not look completely natural.

Below are two different examples:





As you can probably see in the overlays, idle motion from the second motion is naively copied over. Ignoring idle motion by using some type of cropping or time alignment would alleviate such problems.

Until Next Week:
I will continue to flesh out the tool to allow more options in creating the overlays. Once I finish this, I will try to find some more motions that we can use to test the overlays.