While we work on trying to fine tune the physics checks, I also have to think about other ways to improve the overlay motions...
My primary goals right now are:
Create the game environment in OGRE
Implement a method of shoulder alignment using point clouds to improve the overlays, which is described in "Splicing Upper-Body Actions with Locomotion" by Heck, Kovar, and Gleicher (I'm sure I mentioned this in an older post as I've been trying to find time to do this for a while).
I am hoping to get a preliminary version of the game up soon. Once that is done and assuming things go fairly smoothly, I'll be able to see how the overlays motions I generated look on another character model.
I previously wrote a script to automatically generate overlays given the directories of motions desired for the lower body and upper body actions. The next step was implementing constraints to filter the results.
I integrated our physics checker (thanks to Aline) with the generation script to denote whether or not a created overlay is physically correct. Of course, the checker is not perfect, and even if it were, physically correct overlays are not guaranteed to look realistic or natural. For now, we're using it as one step in the right direction for generating better overlays.
With this script, I can generate a lot of different overlays. Currently, I'm examining which overlays I think look good (does it look like something a human would realistically do?) and why some others do not look as good.
I improved the overlay creation tool to allow the user to transplant individual limbs. By default, all the limbs above the root will be transplanted. However, the user can select and de-select individual limbs. The interface is also very easy to use, allowing you to toggle between transplanting an entire arm. To demonstrate, here is a more recent video of the overlay creation process.
The user first loads the motion he/she wants to use for the lower body. Then the user selects to create an overlay and loads the motion for the upper body. Options include interpolating between the two motions before transplanting and selecting individual limbs as mentioned before.
Here is an example of the difference between an overlay where all the upper body limbs are transplanted and one where one of the arms is not transplanted:
In the example above, I am obviously using a running motion for the lower body and a drinking motion for the upper body. In the first overlay, all upper body limbs are transplanted from the drinking motion to the running motion. As a result, the right arm remains stiff because it wasn't moving in the original drinking motion. However, if we choose not transplant this arm we get the result shown in the second overlay, which I think looks a bit better.
Transplanting individual limbs can really help make better looking overlays. Of course, it also multiplies the number of possibilities we have to consider when thinking of ways to automatically generate overlays.
I previously implemented the ability to generate multiple overlays through the GUI. I decided to also create a standalone script to generate the overlays and parse through multiple directories of lower and upper body motions. This should be useful and quicker than having to rely on doing it through the motion viewer program.
Meanwhile, I've been studying previous works in closer detail. I will probably post a detailed review of related papers as the project continues.
I thought I'd do a quick post about what I'm working on now and planning for the next few weeks, as well as the remainder of the project.
Thorough review of related work - I reviewed previous work when I first proposed the project and have found other relevant work as I've gone on. I'm going to through all the work again to determine the best direction to go in as I get into the major part of the project.
Implement constraints - So far, I'm only generating overlays without constraints. The next step will be to implement constraints when automatically generating overlays, primarily physics checks for now.
Analyzing results - After the constraints are implemented, I will thoroughly analyze the resulting overlays to see what works and what can be improved.
Game environment - I'll create a game environment in OGRE to visualize the overlays through a game character.
I'm going to deviate from my original timeline a little bit. I don't think I'll spend nearly as much time on the game portion as I originally thought. Instead, the game environment will primarily be another means of viewing the overlays, and hopefully will make the results look more interesting. I'll be focusing much more of my time in analyzing the overlays and trying different methods to improve the results when automatically generating them.
The key part of this project is generating a large number of overlays and analyzing them. I implemented a feature in the user interface where you can generate overlays by specifying one folder of lower body motions and one folder of upper body motions.
I thought this would be a useful way to generate overlays especially when I implement constraints. This can easily be extended into a console application to generate a massive amount of different combinations.
We can quickly inspect the overlays once they are generated. I structured the output so that every upper body motion has a different folder, and the files within it are named lowerbodymotion_upperbodymotion.amc with their respective names.
We noticed that there was some twitching in some of the overlays. After observing and playing around it with it a bit, I found that the problem lied in transplanting everything above the root. Update: Ignoring the lower back joint and transplanting everything above it looks better but loses some of the movement of the motion.
I'm also keeping in mind other methods of alignment, namely spatial alignment mentioned in the paper "Splicing Upper-Body Actions with Locomotion" by Heck, Kovar, and Gleicher. This involves finding a local rotation of the pelvis that best aligns the shoulders and spine of the upper body motion with that of the lower body motion. The best rotation is computed using point clouds.
Update: Something like the method mentioned above seems like a good way to start.
