InverseKinematics | Dencho

Physics Based Procedural Animations [02]

Wed, 16 Nov 2022 00:00:00 +0000

Calibrating the muscle joints on big Boss Character.

This character is for a physics based combat system I cant exactly talk about due to an NDA.

For calibration and debugging physics I setup quick disposable testbed logic that might look pretty weird out of context.

When done right, this guy will struggle to move objects with heavier mass than others. This is what it looks like when moving all objects with the same mass.

And this is why we calibrate.

Crypto Comedy Club

Wed, 05 Jan 2022 00:00:00 +0000

Contracted out by a small team with plans to launch a Comedy Club in the Metaverse tied to custom NFT metadata and the $LOL token.

Devices tested:

Oculus Quest 1/2
Pico 2/3

Technologies used:

VRoid SDK (Avatars)
ReadyPlayerMe SDK (Avatars)
OculusVR SDK
UnityXR SDK
PicoVR SDK

Due to an NDA im not legally allowed to show the NFT portion of this project.

The BlockChain

Implementing the blockchain took some work, however at the time we got it working due to an SDK called Moralis that was coming out at the time. Token and NFT transactions were able to take place using the connected Web3 wallet.

One of the main features is being able to hop into a theatre and watch a comedian perform a live or recorded session with your friends.

Physics Based Hand-Tracked Posing

One of the main problems with HandTracking is the immersion break a user experiences when they reach out to grab a virtual object. Usually the virtual hands that mimic the real world hands end up clipping through the object the user is grabbing. This is my attempt to fix that.

During my time at CCC I prototyped procedural hand poses when grabbing things with pure hand tracking. Each fingers Inverse Kinematic system is programmed to bend until a collider is detected.

Working with ReadyPlayerMe Avatars

The final PlayerController looked something like this with a live RPM Avatar.

With both Hand and Controller tracked interaction!

The UI eventually worked well enough.

Good enough to test out Peer2Peer Networking for the first time!

However there were some bugs involving Physics that didn't go well over the network.

Audience Taming

Stress testing remote player count. Turns out even low quality avatars were such a big performance hit.

So I just used voxelized avatars for the audience which solved the performance hit from a full house.

XR Player Controller [05]

Sat, 15 May 2021 00:00:00 +0000

Refining one of the first Prototypes to exist (in 2020) to contain a full body avatar and finger tracking.

Devices tested:

Oculus Quest 1/2
Pico 2/3

Technologies used:

VRoid SDK (Avatars)
ReadyPlayerMe SDK (Avatars)
OculusVR SDK
UnityXR SDK
PicoVR SDK

First thing I did this round, was fail hard at using Mechanim Animator to handle my Hand Posing. Turns out its just easier and more performant to man handle hand posing with pure C#.

Semi working legs. Check.

You know I had to make sure everything worked with physics. Just incase.

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

S.M.O.M. happens when one mesh skeletal bone structure has incompatible orientations with another set of bones, leading to misalignment when trying to animate or manipulate them consistently.

You can expect to run into SMOM when using ReadyPlayerMe avatars for a full body rig.

However, SMOM is easily defeated with the use of my custom Hand Mapper.

The results, a full body XR controller that dynamically switches between hand and controller tracking. Not something you see often in 2021.

And a simple voice plugin test to move the avatars facial blendshapes.

During development, this gave me nightmares.

Physics Based Exo-Skeleton Arms

Tue, 28 Apr 2020 00:00:00 +0000

I believe VR can be a cheap and effective way to test out ideas. On day I wondered how fun it would be to give the user giant Exo Arms.

This crude prototype gave me decent understanding one how it would feel for one arm anyway.

XR Player Controller [04]

Fri, 10 Apr 2020 00:00:00 +0000

Refining one of the first Prototypes to exist (in 2020) to contain a full body avatar and finger tracking.

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

VRoid SDK (Avatars)
OculusVR SDK
UnityXR SDK

I was one of the first developers in the industry to achieve fullbody avatars working with finger tracking. Once you get past the hurdle of different bone orientations the rest is pretty simple, I was just a dev with just enough free time to actually do it.

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

Using this formula here I was able to avoid this issue, however the custom tool I made requires a basic understanding of how Quaternions work.

Testing UI Ray Pointers with Finger Tracking.

I wrote an algorithm that tracks and predicts the tracked hands position and velocity. My goal was to Handle the case of when Hands overlap causing hand tracking to be lost due to occlusion. This algorithm understands when the Hands become occluded and attempts to predict and move the hands relative to their original trajectory at the time of occlusion.

You can see that since I am simply lerping the predicted target IK position of each hand, there is no teleporting going on when hand tracking resumes from being occluded.

When Hand tracking quality becomes low, mapping of the fingers halt and then blend to the target orientations once tracking quality is high again.

Testing relations with Physics based Grabbables when a Button magically resets an objects transform.

When this Physics button is pressed the Cubes Reset.

Nothing much to take from this other than knowing the rigidbody wont explode the Players rigidbody in some way.

There were a few setup issues with the relative Rig height. Nothing I can't handle.

Physics Based Exo-Skeleton Arms

Wed, 18 Mar 2020 00:00:00 +0000

This was one of the most difficult prototypes for my skill level at the time. The prototype code would need to be refactored before I would attempt putting it into beta testing.

Part of the math I needed to master was how 3D normals and reflections are calculated.

Yes its a Sphere for now, but keep in mind this prototype is using RigidBodies and a Velocity PID controller that I wrote to move the body via Physics.

Testing Variable Properties out.

Physics Based Buttons [02]

Wed, 11 Mar 2020 00:00:00 +0000

Another contract to improve front-end buttons. NDA prevents me from showing the exact implementation of the new buttons but, I am allowed to show them off on my own Menu Prototype.

XR Player Controller [03]

Tue, 10 Mar 2020 00:00:00 +0000

One of the earliest prototypes to exist at the time (2020) that I know of, to contain a full body avatar and finger tracking.

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

Vroid SDK (Avatars)
Oculus VR SDK
UnityXR SDK

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

S.M.O.M. happens when one mesh skeletal bone structure has incompatible orientations with another set of bones, leading to misalignment when trying to animate or manipulate them consistently. It’s essential to ensure that both hand meshes follow the same bone orientation conventions to avoid issues during animation and rigging processes. However sometimes this cant be avoided when sourcing from different third parties, in this case I had trouble aligning VRoid Hand Skeleton to match with the Hand Skeleton from the OVR SDK used for hand tracking.

Bypassing S.M.O.M.

When dealing with skeletal mesh orientation mismatches between two hand skeletons, you can use a transformation formula to align their orientations. This approach allows you to transfer the orientation from one hand to another, even if they initially have different orientations.

Using this formula here I was able to avoid this issue, however the custom tool I made requires a basic understanding of how Quaternions work.

Using a Homemade VRoid Avatar, which uses a Universal Skeleton Armature. This is what happened when trying to sync up rotations from OVR custom hand skeleton to the VRoids hand.

Locomotion Testing

Locomotion Jumping with Physics still needs work.

Physics Based Procedural Animations [01]

Tue, 18 Feb 2020 00:00:00 +0000

I love working on physics based animations.

I love procedural animations, most system use Inverse Kinematics to accomplish this, as do my systems. However I wanted to take it to the next level and allow dynamic blending between Inverse Kinematics and Physics Simulations.

This should allow my to play a Humanoid animation on the skeleton while still applying IK to the bones, and lerp between a "Live" and "Ragdoll" state for the characters physics body.

Its far from perfect and I still need to find a use case but, its still fun to look at.

Physics Based Buttons [01]

Tue, 11 Feb 2020 00:00:00 +0000

Contract to improve front-end buttons. NDA prevents me from showing the finished prototype.

XR Player Controller [02]

Mon, 10 Feb 2020 00:00:00 +0000

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

Vroid SDK (Avatars)
Oculus VR SDK
UnityXR SDK

This was my second attempt at a XR Player Controller. My main goal here was to make sure I can modularize components while solidifying most of the prototype code.

The Avatar was home made this time using a base Vroid from Vroid Studio and Decimating it in Blender for low poly testing.

I wrote the Grabbable components you see here, in order to make it work with physics.

Held hand poses still need some work.

Testing some facial blendshapes out.

Jumping still needs work.

HandTracking was a complete failure as different avatar rigs have skeleton bones that use a different axis orientation. So one can not simply match up rotations 1:1 unless all bones use the same orientation axis.

XR Player Controller [01]

Fri, 10 Jan 2020 00:00:00 +0000

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

UMA2 (Avatars)
Oculus VR SDK

This was my first attempt at a XR Player Controller. Starting small my goal was to just get each limb lined up with basic controller based Hand Poses working.

Clothes Placements and RayPointers

My own Custom Grabbables and my try at 2 handed grabbing.

Writing my own Locomotion was great practice. More work needed to be done on the jumping.