Nice to meet you
Just call me Jesse.
I'm a technical User Experience designer based in Chicago, being passionate about XR design and other tangible IXD.
What if there was a system in place to support hybrid living?
HMW connect hybrid workers with the right resources in their neighborhood.
When WebXR becomes common, how should designers get ready for the new design area?
Getting more people involved in Translational Research is the key to save more lives.
In the new norm, HMW bring back daily socialization for remote workers while protecting their boundaries?
In the post-pandemic, are cloud drinking and in-home dancing possible? MR gave the solution.
What if facial expressions are a new input for games? How life looks like in AR.
Summer Intern in iMotions, working on implementing eye-tracking SDK to Varjo XR-3 and making tutorials for clients and sales team.
What's the user experience when Lvl 5 AV comes to campus? An entire system for self-driving shuttle.
Using COVID daily death data , bringing a new Monument to tell one thing — the passing lives are not numbers, we should remember them.
Animation, Game design
An experimental puzzle game built by ThreeJS, aiming to explore the capability of WebXR API.
Block Puzzle Game
Z L T
Challenge & Outcome
ThreeJS is totally a new framework to me, I have no background about 3D modeling, let along coding and rendering 3D elements. Meanwhile, ThreeJS only provides trigger and squeeze events. Other controllers events I have to refer to W3 XRInputSource. However, I almost figured out how the 3D Objects work here and understand how to get other input sources in the Oculus Quest test environment at the end.
Sorry about the typo! I was too excited to check the language when I completed the game and directly recorded the video... 😅
Player needs to use the appropriate block to make shapes that can be matched to the given ones on the wall. There’s neither any punishments if you can’t make it till the end, nor any rewards if you complete levels. Only thing you need to do is to focus on the jobs at hand, trying to challenge your talents and aftertaste your childhood with endless imagnations. Once the game is completed, please enjoy the world you created.
For the first ideation, I created the sample scenes in Tiltbrush, which could fully express the design in VR.
Mouse drag to change camera perspective angle.
thumbstick on the left controllers to move around in VR
Mouse click to select blocks and place them on the floor
trigger in VR to select blocks and place them on the floor
Select + Place
Shift + click to delete the one you want to remove from the floor.
trigger + squeeze in VR to delete the one you want to remove from the floor *when there’s one block selected
Left and right arrow keys to rotate the selected block along its own given axis
thumbsticks on the right controller in VR to rotate the selected block along its own given axis
As a 3D webGL game, space is important. Players could have different and amazing experience when they are having fun in a real-like space. The effect on web should be familiar with the one in VR experience. So I choose Perspective Camera, giving the rights for players to explore around on the desktop.
In VR world, Teleport is a good choice to reduce dizziness, but it is not fit in block game. Because players needs to adjust their position they feel good to understand the puzzle. So I apply thumbstick to let them move in the new world (on the ground with some boundaries)
Physical feedback is the last thing we would give up in our real world. In the digital VR world, except audio feedback, visual effects are also an efficient way to make up physical feedbacks.
I provide several visual effect to let player they are on the right way.
Select blocks to place.
Wrong color to say it is not right
Flashing shadows to tell this is the new level.
Final animation to provide a sense of accomplishment.
Reading instruction on web is easy for most of people, but in the VR world it is hard for designers and developers to set the right size. So I place the words on the left controller, players can read it as reading watch.
Have an array in the loop to read the "1" can draw the block. Then re-center the position by its bounding box in order to have the right WorldPosition for later use.
I don’t let the computer read the centers of each cube every time, so I recreate shadow pieces which can be remove and add based on the rotation of block.