Augmented Reality Map Application - part 2
Nov 5, 2019
My blog post about the Augmented Reality Map Application had several very interesting conclusions.
One was that I wanted to generate the 3D scene by Shape Builder. The logic behind this is that the application size could be reduced and it could scale as well for other scenarios. Although the geometry would be hand-coded, a rectangular board and a big number on it didn’t seem to be too complicated. I have the code for that still in a branch. The numbers could be textures of the billboards. The images could become large if we resorted to high resolution; in the case of low resolution, the large board could reveal pixelation of the texture. But one 3D board geometry could act as any sign: we could just replace the textures.
Unfortunately, that didn’t work. Nothing showed up, and I’m not sure if it was due to the giant texture resolution or not. I almost got stuck, so to get that part of the job done, I learned Blender in an extremely short amount of time and developed each board one by one. Each board has its own number as a 3D extruded object on it: a red board with a white number so it would be very visible. The downside of this approach is twofold: a) I had to design every board in Blender, and that wouldn’t scale well for a bigger building (note that I needed to generate an sfa and sfb for each and every board: Model directory). b) The extruded numbers can pose 3D computational pressure if their tessellation is too refined (in layman’s terms, if the curves of the number 3, for example, are segmented into too many sections). In any case, I got this part of the job done.
The second was a feature I ended up dropping: a compass pointer to a selected billboard. In my original design, I would have listed all the billboards in a bottom list view, and the user could make their choice. Since the user might be facing away from the billboard’s location at the time of selection, it would be ideal if a floating compass that is always in front of the user’s view pointed towards the desired billboard. I produced the 3D geometric calculation for that and also derived the 3D model from an arrowhead, but somehow the compass worked funky. I didn’t have time to debug this after midnight, so for now it’s dropped, but I plan to bring it back.
In the future, if there’s a conference and the user logs in to the app (right now the app doesn’t have any authentication), we could examine the user’s schedule and automatically focus on the right billboard representing the user’s scheduled class for the given time. This could be a future capability as well in case the app is used as a conference companion.
The app can be a standalone venue helper as well. Larger venues could have many more activator photos. Each of those photos would be located in a well-known position, and the app could list the closest locations or the user could browse all listings. Since I talked about scalability earlier, such a version would really require the textured billboard method. In such a case, the app could dynamically load the list of POIs (Points of Interest) from an online database (the app could have a default set in case the user is offline, and fresh updates could come through online). This way, the app’s capability could be administered dynamically!
Here is my presentation about the app: Applied Augmented Reality with Sceneform SDK.
SceneForm really brought the Augmented Reality game to a higher level where the entry point doesn’t require too complicated geometric knowledge (unless you want some extra things). Such moves are required for Augmented Reality to finally have a real breakthrough. One extra thing to check out in my app is the logic which causes the billboards to automatically rotate towards the user. If that wasn’t done, the user could walk around the billboard, and if viewed from the side, the user would just see a red line. During the C-64 era—before GPUs really brought enough computational power to computers—such boards were actually easier to implement: you just had to scale the texture and not worry about complicated projections to the user’s viewpoint. Nowadays—since SceneForm is full 3D all around—we need to apply some extra logic to achieve that simple "billboard" effect. See the billboard logic here.
Here is the "promo" video showing the app in action:
This browser does not support the YouTube video player. Watch on YouTube
Valley DevFest 2019 Augmented Reality Map demo and promo video.
Here are some more pointers: