Problem
This project posed many UI/UX questions, augmented reality has many interaction paradigms.
AR requires the user to not only use traditional touchscreen inputs, but doing so while holding the device steady, physically move it around to track virtual objects, and keep the camera steady so they can perform touch gestures.
Our job was to develop an app that not only did this successfully, but create a 3D avatar with facial and object recognition as part of its core interactions with the player.
Process
We first explored augmented reality by creating an in-person guided experience that debuted at the Sundance Film Festival.
During initial prototyping to alpha my responsibilities lay in developing the main Guppy character. Including art, procedural animation, UX, UI, and C# logic.
By creating a guided app we were able to learn that many of the major pain points had to do with users moving extremely quickly with the phone, tracking loss, lighting conditions, and room setup.
We created a snappier experience app that was more about exploring, not staying still in one place, that way if tracking broke it was not an issue, it was more about moment to moment interactions.
Mitch developed the narrative systems driving Guppy that were then connected to the minigames and interactions on my end.
Throughout this process we were supported by Julian to implement AR in a performant and reliable manner.
Outcome
Tendar successfully established the Tender Claws studio and was nominated for a Games for Change award.
We safely deployed three new technologies: augmented reality, face, and object recognition, all in a single experience.
The narrative scripting language for the project (a markup language to help develop Markhov chain based narratives) was also open-sourced.
