I've been listening to Dungeon Crawler Carl recently and it inspired me to make a game. What if you could play an RPG in AR? With friends?
What could be better than running around your neighbourhood with your phone in front of your face, sweating like a pig, frantically swiping at monsters that don't exist? This is a good idea, right? RIGHT?
Anyway, I slogged through making a playable proof of concept, and I'm pretty happy with where it's landed. This is a screen recording from my Galaxy S24: find the Ugly Stick, hear two goblins arguing in the distance, get too close, fight them, loot the bodies and collect an achievement.
The video is the first single-player scene. The bigger vision is a season-based, multiplayer, real-world RPG: parties exploring parks and trails, finding loot and surviving encounters, with an entertaining System AI handling mob descriptions, loot and achievements. I'd eventually love to bring it to AR glasses too.
It's built in Unity with AR Foundation/ARCore. Claude and Codex helped build and debug the encounter logic, HUD, AR grounding and camera stabilization with a bunch of testing. I also automated a monster creation, skinning, rigging, animation and narration pipeline using TRELLIS.2, SkinTokens, Gemini and ElevenLabs.
Getting the creatures grounded and the camera steady while walking took some work. Stabilizing the finished recording also made the HUD wobble, so I moved stabilization into the app, before the HUD is drawn. Ground height also refines as you approach the creatures.
I'd love feedback from people building or playing handheld AR games: how does the grounding look, and how would you make the combat easier to read while walking?
I'm also looking for one or a few nerds who think this is neat and want to contribute, ideally someone who could help lead the project. I can bring the prototype, asset pipeline, design input, playtesting and too many ideas. Work and kids mean I can't drive the whole thing myself while also trying to keep my marriage from falling apart.
If you have suggestions, know someone who might be interested, or want to help build it, I'd love to hear from you.