Experiments | Dencho

VFX Practice [07]

Fri, 10 Feb 2023 00:00:00 +0000

A small experiment to create a reactive shadergraph with exposed properties that a custom controller can use to make the shader react to the beat of an audio clip.

And a VFXGraph that controls particles to move along an array of world points over time. In this case I calculated a Hexagons orientation and size from a cells center position, then fed each corner to the VFXGraph via my custom tool.

Physics Based Procedural Animations [02]

Wed, 16 Nov 2022 00:00:00 +0000

Calibrating the muscle joints on big Boss Character.

This character is for a physics based combat system I cant exactly talk about due to an NDA.

For calibration and debugging physics I setup quick disposable testbed logic that might look pretty weird out of context.

When done right, this guy will struggle to move objects with heavier mass than others. This is what it looks like when moving all objects with the same mass.

And this is why we calibrate.

Terrain Practice [01]

Fri, 11 Nov 2022 00:00:00 +0000

Frustum Culling is a technique used in computer graphics to improve rendering performance by discarding objects that lie outside the viewing frustum.

Frustum culling is a powerful optimization technique that balances computational cost with rendering efficiency. This is also a technique used in popular AAA titles like CyberPunk and Zero Dawn Horizon.

Terrain Practice [02]

Fri, 11 Nov 2022 00:00:00 +0000

Frustum Culling is a technique used in computer graphics to improve rendering performance by discarding objects that lie outside the viewing frustum.

XR Player Controller [05]

Sat, 15 May 2021 00:00:00 +0000

Refining one of the first Prototypes to exist (in 2020) to contain a full body avatar and finger tracking.

Devices tested:

Oculus Quest 1/2
Pico 2/3

Technologies used:

VRoid SDK (Avatars)
ReadyPlayerMe SDK (Avatars)
OculusVR SDK
UnityXR SDK
PicoVR SDK

First thing I did this round, was fail hard at using Mechanim Animator to handle my Hand Posing. Turns out its just easier and more performant to man handle hand posing with pure C#.

Semi working legs. Check.

You know I had to make sure everything worked with physics. Just incase.

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

S.M.O.M. happens when one mesh skeletal bone structure has incompatible orientations with another set of bones, leading to misalignment when trying to animate or manipulate them consistently.

You can expect to run into SMOM when using ReadyPlayerMe avatars for a full body rig.

However, SMOM is easily defeated with the use of my custom Hand Mapper.

The results, a full body XR controller that dynamically switches between hand and controller tracking. Not something you see often in 2021.

And a simple voice plugin test to move the avatars facial blendshapes.

During development, this gave me nightmares.

AI Mob Controller [05]

Mon, 10 May 2021 00:00:00 +0000

This experiments goal was to iterate on my previous AI Controllers and refine the locomotion and procedural animations.

I spent a good amount of focus on the making sure procedural animations work with physics.

The result allowed me to play an animation, and have physics forces affect it in a nice good lerpy way.

Next I applied the same system to another nonstandard skeleton.

Dynamic Collider System

Wed, 05 May 2021 00:00:00 +0000

Colliders that transform with Physics.

The goal of this system was to make hit events more generous relative to the movement of the rigidbody attached to the HurtBox.

To start I made some gizmos that show the thresholds of the dynamic settings you set in the component.

This basically works as a Melee auto aim for a XR perspective. If the player thrusts a knife forward sometime VR depth can take getting used to, this makes it easier for players to hit something that might be a few inches father then perceived.

It actually works out pretty well, and I wrote it to be well performant. Just attach to a meshs child collider (box type) that has a rigidbody.

XR Bridge System

Thu, 15 Apr 2021 00:00:00 +0000

The goal of this experiment was to create a procedural dynamic map out of bridges and hub nodes of my own design.

One problem I encountered was during the Spawning/Despawning of the bridges, framerate would drop on the Quest1 due to performance issues with colliders moving over time. This meant I could only make bridges so long before frames would skip.

It worked well in theory but, in practice it's not that feasible on a low powered device like the Quest1.

XR Magic System

Thu, 15 Apr 2021 00:00:00 +0000

I wanted a modular magic system that works with physics based grabbables.

So I made one, it has public methods that allow the user (mob or player) to charge mana into the magic spell that is being cast at the time of input.

Once I got spawning pretty good, I added Telekinesis control, after a grabbable is spawned it will track the users appendage until mana is no longer being channeled into the spell.

I have no idea why I started making this, it just seemed fun. I hope to iterate upon it in the future.

AI Mob Controller [04]

Sat, 10 Apr 2021 00:00:00 +0000

This experiments goal was to iterate on my previous AI Controllers and refine the combat system.

I worked more on the combat skill pathing. It is now more dynamic while still using a physics PID controller to move the rigidbody via forces in additive manner.

Both of these mobs use the same combat system, but one is tweaked to have a shorter travel path, with drag/angulardrag over time (set by a curve and multiplier) which allows me to create something like a bulletTime attack but in realtime.

This kind of system allows other physics objects to alter the pathing trajectory of the mob even while the skill is running.

Even works with drone like locomotion.

Discord RPC

Mon, 05 Apr 2021 00:00:00 +0000

Custom Discord Plugin for Unity.

A client requested integration with the Discord API, at the time there was no official Discord SDK in C#, I endedup using a Discord API C# wrapper I found on GitHub.

Its just a simple plugin you can import, that hooks into a running discord client and shows a custom activity based on a scriptableObject config you can setup. It works in the Editor and during runtime and can be extended on to use analytical funnels.

Fun With Gizmos And Math

Thu, 18 Mar 2021 00:00:00 +0000

I create alot of small gizmos extensions, some of which for no reason and for fun.

This was for debugging AI and Player Input Intent direction and overlaying angle thresholds to visualize the logic state during runtime.

Look familiar?

VFX Practice [06]

Wed, 10 Mar 2021 00:00:00 +0000

More VFX practice so that I might one day be able to make a decent looking kamehameha.

I made a custom Particle Effector that works with VFXGraph to control where the particles go in realtime.

And a laser, cause why not?

AI Mob Controller [03]

Wed, 10 Feb 2021 00:00:00 +0000

This experiments goal was to iterate on my previous AI Controllers and refine the spawning and path finding algorithms.

The models were provided by free assets on the Unity Asset Store.

Created a Waypoint system that only allows one mob at a time to navigate to the point, the mobs automatically seek out point in a waypoint path that doesn't have its source/target points queued by another mob.

Testing out Ragdolling system for a nonstandart mesh rig.

A Combat system that animated the Rigidbody with a Physics PID Controller to traverse a dynamic generated path, with adjustable curves/properties to change up how it ride the attack path.

AI Mob Controller [02]

Sun, 10 Jan 2021 00:00:00 +0000

This experiment is mainly focused on ground based AI designed to work with a XR Player Controller.

First I wrote my own PID controller to control a Rigidbody via physics to match a point in world space with gravity accounted for.

Combat Testing. Just going to throw a rock at it.

Stress-testing spawn mechanics.

GameJam [01]

Fri, 12 Jun 2020 00:00:00 +0000

This GameJams theme was the exploration of destructible objects using magic.

The first goal was to code out a Telekinesis framework that uses a signle input to lift and throw physics bodies

Next was to make the same TK system affect structures connected by configurable joints.

Next I wrote a tool to automagically detect child rigidbodies and attach recursive connecting configurable joints to each parent/child combo according to a configuration set before runtime. Pain to write, but satisfying to test.

Next came alternate TK abilities and properties to achieve different features from the same framework.

Sadly we ran out of time, I learned in the future to make sure to work with a more capable team.

VFX Practice [05]

Wed, 13 May 2020 00:00:00 +0000

Rhythm based VFX, A simple test of animating VFX components to the beat, volume, and onset of an analyzed audio clip.

Turns out everything works which means I can make a rhythm game now.

Next I'll try making this work with VFXGraph and ShaderGraph.

Physics Based Exo-Skeleton Arms

Tue, 28 Apr 2020 00:00:00 +0000

I believe VR can be a cheap and effective way to test out ideas. On day I wondered how fun it would be to give the user giant Exo Arms.

This crude prototype gave me decent understanding one how it would feel for one arm anyway.

XR Chemistry Framework

Sat, 18 Apr 2020 00:00:00 +0000

I was contacted and contracted out by a School District Manager looking for ways to Adapt XR as a means for kids to learn remotely in a more immersive way.

The things I can show from this project are limited due to an NDA

The main goal of this prototype was to create a UI capable of spawning Chemistry related instances of Molecule models and allow interaction with them for lecturing purposes in a Teacher to Student relationship.

As the project was already 12 months into development at the time I joined, I had to work well with the existing team to know where to put my code and to make sure there were no conflicts with existing UX components

Prototyping a way you can move molecules with hand tracking, even under low lighting.

The idea was that if a hand pinches near the center of a molecule group, the molecule with use the relative index finger as a Anchor with a Kinematic PID controller handling the positioning.

First I start with a Transform Marker in the form of a filling sprite, that acts as the target position for the molecule.

Now just a simple controller that takes account of the distance between the current and target points. Its always fun to see how well I can write something the first time.

Adding a Highlight to show when a molecule is a valid target to grab.

Then I made similar control method for orientation

The UX needs to be cleaned up a bit, but all in all it was a success.

VFX Practice [04]

Mon, 13 Apr 2020 00:00:00 +0000

This was made using ShaderGraph for Unity and some C# elbow grease.

XR Player Controller [04]

Fri, 10 Apr 2020 00:00:00 +0000

Refining one of the first Prototypes to exist (in 2020) to contain a full body avatar and finger tracking.

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

VRoid SDK (Avatars)
OculusVR SDK
UnityXR SDK

I was one of the first developers in the industry to achieve fullbody avatars working with finger tracking. Once you get past the hurdle of different bone orientations the rest is pretty simple, I was just a dev with just enough free time to actually do it.

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

Using this formula here I was able to avoid this issue, however the custom tool I made requires a basic understanding of how Quaternions work.

Testing UI Ray Pointers with Finger Tracking.

I wrote an algorithm that tracks and predicts the tracked hands position and velocity. My goal was to Handle the case of when Hands overlap causing hand tracking to be lost due to occlusion. This algorithm understands when the Hands become occluded and attempts to predict and move the hands relative to their original trajectory at the time of occlusion.

You can see that since I am simply lerping the predicted target IK position of each hand, there is no teleporting going on when hand tracking resumes from being occluded.

When Hand tracking quality becomes low, mapping of the fingers halt and then blend to the target orientations once tracking quality is high again.

Testing relations with Physics based Grabbables when a Button magically resets an objects transform.

When this Physics button is pressed the Cubes Reset.

Nothing much to take from this other than knowing the rigidbody wont explode the Players rigidbody in some way.

There were a few setup issues with the relative Rig height. Nothing I can't handle.

Physics Based Exo-Skeleton Arms

Wed, 18 Mar 2020 00:00:00 +0000

This was one of the most difficult prototypes for my skill level at the time. The prototype code would need to be refactored before I would attempt putting it into beta testing.

Part of the math I needed to master was how 3D normals and reflections are calculated.

Yes its a Sphere for now, but keep in mind this prototype is using RigidBodies and a Velocity PID controller that I wrote to move the body via Physics.

Testing Variable Properties out.

VFX Practice [03]

Fri, 13 Mar 2020 00:00:00 +0000

This was made using VFXGraph for Unity.

XR Player Controller [03]

Tue, 10 Mar 2020 00:00:00 +0000

One of the earliest prototypes to exist at the time (2020) that I know of, to contain a full body avatar and finger tracking.

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

Vroid SDK (Avatars)
Oculus VR SDK
UnityXR SDK

Skeletal Mesh Orientation Mismatch (S.M.O.M.)

S.M.O.M. happens when one mesh skeletal bone structure has incompatible orientations with another set of bones, leading to misalignment when trying to animate or manipulate them consistently. It’s essential to ensure that both hand meshes follow the same bone orientation conventions to avoid issues during animation and rigging processes. However sometimes this cant be avoided when sourcing from different third parties, in this case I had trouble aligning VRoid Hand Skeleton to match with the Hand Skeleton from the OVR SDK used for hand tracking.

Bypassing S.M.O.M.

When dealing with skeletal mesh orientation mismatches between two hand skeletons, you can use a transformation formula to align their orientations. This approach allows you to transfer the orientation from one hand to another, even if they initially have different orientations.

Using this formula here I was able to avoid this issue, however the custom tool I made requires a basic understanding of how Quaternions work.

Using a Homemade VRoid Avatar, which uses a Universal Skeleton Armature. This is what happened when trying to sync up rotations from OVR custom hand skeleton to the VRoids hand.

Locomotion Testing

Locomotion Jumping with Physics still needs work.

AI Mob Controller [01]

Tue, 18 Feb 2020 00:00:00 +0000

The goal of this experiment was to learn what it takes to create an AI that operates within perspective of a XRPlayerControllers camera view.

First I wrote my own PID controller to control a Drones Rigidbody via physics to match a point in world space.

Adjusting the variables allowed to to appear more smooth and animated.

Different properties can lead to different behaviors.

Now we give it a gun. For learning.

Make sure it knows how to shoot. With even the recoil effecting the PID controller.

This can get pretty chaotic but, its good to know how many bodies I can simulate at a given time when you consider theres a PID controller for each Rigidbody.

Physics Based Procedural Animations [01]

Tue, 18 Feb 2020 00:00:00 +0000

I love working on physics based animations.

I love procedural animations, most system use Inverse Kinematics to accomplish this, as do my systems. However I wanted to take it to the next level and allow dynamic blending between Inverse Kinematics and Physics Simulations.

This should allow my to play a Humanoid animation on the skeleton while still applying IK to the bones, and lerp between a "Live" and "Ragdoll" state for the characters physics body.

Its far from perfect and I still need to find a use case but, its still fun to look at.

VFX Graph Projectile [02]

Sat, 15 Feb 2020 00:00:00 +0000

One of my goals was to make the projectile a grabbable with dynamic states, meaning it could be held and thrown like a grenade, or charged and shot like a plasma blast.

Avatars provided by the VRoid Hub.

This allowed me to test out some Combat simulations

VFX Practice [02]

Thu, 13 Feb 2020 00:00:00 +0000

This was made using VFXGraph for Unity.

XR Player Controller [02]

Mon, 10 Feb 2020 00:00:00 +0000

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

Vroid SDK (Avatars)
Oculus VR SDK
UnityXR SDK

This was my second attempt at a XR Player Controller. My main goal here was to make sure I can modularize components while solidifying most of the prototype code.

The Avatar was home made this time using a base Vroid from Vroid Studio and Decimating it in Blender for low poly testing.

I wrote the Grabbable components you see here, in order to make it work with physics.

Held hand poses still need some work.

Testing some facial blendshapes out.

Jumping still needs work.

HandTracking was a complete failure as different avatar rigs have skeleton bones that use a different axis orientation. So one can not simply match up rotations 1:1 unless all bones use the same orientation axis.

VFX Graph Projectile [01]

Wed, 15 Jan 2020 00:00:00 +0000

I made this projectile using a home made VFX graph, and some C# elbow grease to make the projectile act like its part of a physics based charging system.

Shooting some boxes. Yes I am aware im shooting the projectile out of a scope.

The idea behind this kind of system is the modularity of the script I made allowing it to work with any VFXGraph setup with the right event and exposed properties.

Avatars provided by the VRoid Hub.

VFX Practice [01]

Mon, 13 Jan 2020 00:00:00 +0000

Some custom code to rotate simulated points and emit particles offset via some maths.

This was made using VFXGraph for Unity.

XR Player Controller [01]

Fri, 10 Jan 2020 00:00:00 +0000

Devices tested:

Oculus CV1
Oculus Quest 1

Technologies used:

UMA2 (Avatars)
Oculus VR SDK

This was my first attempt at a XR Player Controller. Starting small my goal was to just get each limb lined up with basic controller based Hand Poses working.

Clothes Placements and RayPointers

My own Custom Grabbables and my try at 2 handed grabbing.

Writing my own Locomotion was great practice. More work needed to be done on the jumping.