Its a Date!

https://github.com/its-ajz/Its-a-Date

Romance Choose Your Own Adventure on Apple Vision Pro - Its a Date!

Overview

For our project we made a choose your own adventure experience on the Apple Vision Pro where you go on a date that can go completely off the rails depending on what you pick. You watch immersive 180° video of a date scenario playing out around you and at the end of each scene you get two choices that take the story in a totally different direction. Some endings are sweet, some are chaotic, and one involves a stalker. Pretty unpredictable. The whole point was to make something that felt like you were actually there living the story rather than just watching it.

Development Process

The first big decision was figuring out what to actually build it in. We went back and forth between Unity and Xcode. Unity seemed easier at first since it's more familiar for game stuff, but we ended up going with Xcode and SwiftUI because it has way better native support for Vision Pro and the immersive space API is built right in. That ended up being the right call even though it meant learning more about Swift.

From there the main challenge was just getting all the pieces working together. Getting 180° video to actually play inside the headset took a while. You have to create a sphere mesh, flip it inside out so the texture faces inward, and wrap the video around it so it feels like you're standing inside the scene. Then we had to build the choice UI so it floats in front of you at the right distance when the video ends, wire up the branching logic so each choice loads the correct next video, and set up a JSON file that controls the whole story structure so we could just drop in new videos without touching the code.

On the production side we mapped out the entire storyline on a whiteboard first, figured out all the branching paths, got our cast together, and filmed everything on an Insta360 X4. After shooting we had to export everything in the right format for the headset, equirectangular 180°, mp4,  and then drop the files into Xcode.

Successes

The technical side mostly came together really well. The video plays inside the headset in full 180°, the choices pop up in front of you when each scene ends, and tapping a button actually takes you to the right next video. The whole branching system works cleanly off the JSON file which makes it really easy to swap videos in and out or change the story without rewriting any code. Getting all of that working from scratch on a platform we haven’t worked with much was great!

Challenges

Getting the videos to play at all took a lot of debugging. Wrong formats, files not being found, the sphere showing up black. Figuring out the rotation was tricky because the inside-out scale flip on the sphere interferes with how rotation works, so we had to get a bit creative. Placing the UI in the right spot in 3D space was probably the most frustrating part . Offsets that work in regular SwiftUI don't behave the same inside a full immersive space, and we went through several different approaches before landing on our final version. On the filming side we had to figure out camera height, where to stand relative to the camera, how to keep the stitching clean, and how to avoid the kind of shaky footage that makes people feel sick in a headset.

What We Learned

We learned a lot about Xcode and SwiftUI that we didn't know beforehand. For example, how AVPlayer handles video in the headset. On the filmmaking side we learned how 360° cameras actually capture footage, what equirectangular format means, why camera placement matters so much more in VR than in regular video, and how to export properly for a headset. Building the storyline also taught us how to think about branching narratives. We have to map out every path, making sure nothing dead-ends accidentally, and keeping the scene count manageable while still giving the viewer fun choices.

Future Revisions

There's a lot of directions this could go. The most obvious one is adding more storylines. Different settings, different characters, more branching paths so the experience has more replay value. We'd also want to upgrade to full 360° video instead of 180° so the world feels completely surrounding rather than just in front of you. On the immersion side it would be cool to add spatial audio that actually comes from the right directions, a better resolution for videos, and smoother transitions between videos so the cuts feel less abrupt. Eventually down the line we may add multiple main characters to choose from at the start.