Concept Photo Browser Puts Your Selfies in Augmented 3D Environments

Brett Foxwell's SiteSeer app is like a hypothetical mix of geocaching, Google Street View, and Instagram.

December 4, 2014, 8:30pm

Brett Foxwell's particular area of expertise falls somewhere between engineering and animation, a combination of fields which has led him to develop, SiteSeer, an augmented reality app concept best described as a cross between Google Street View, geocaching, and Instagram. "It is a tool for revisiting and re-experiencing natural locations and viewing photos taken in these locations, a sort of geographic photo browser," Foxwell told The Creators Project. Watch the demo video above to see an imaginary session with the hypothetical photo browsing software.

In the video above, the SiteSeer user explores three different locations: a mountain, a forest, and a desert. Each transforms into a Google Street View-like 3D facsimile of itself IRL. Geo-tagged photos appear within the 3D environment, breaking down the borders of the image to literally give us the bigger picture. Foxwell demonstrates this imaginary functionality with a series of selfies taken on a mountain top, over a bridge, and at the bottom of a desert canyon, seamlessly immersing his photographs inside the 3D-scanned digital environments.

It's important to note, once again, that this is just concept footage; Foxwell manually aligned the images with the digital panoramas, which themselves were hand-scanned and very limited in scope; he connects the digital realms he captured with using hyperlapse footage, which helps SiteSeer transcend the label of mere concept video into something of a standalone short film. Foxwell, however, mostly chose the medium for utilitarian reasons: "They quickly impart a good geographical sense of the environment where they were shot," he said. "So they mix well with the 3D elements."

Whether it's a demo of an app that could be the next Instagram, or a conceptual exploration of potential converging technologies, the fact remains that SiteSeer is certainly easy on the eyes. We spoke to Foxwell about how he came up with the idea for the short, and the challenges of inventing a new medium.

The Creators Project: Where did the idea for SiteSeer come from?

Brett Foxwell: I came up with the idea of an enhanced panorama, which uses a 3D scan of a prominent natural feature. This scan sits at the center of a 360 degree panorama of the feature's surroundings. This can be explored in 3D space and things will line up if the viewer stays close to the center of the panorama. Photos and snapshots showing the feature or the background panorama can be lined up with the 3D elements and the viewer can see the wider landscape beyond the edges of the photo.

Can you tell me about the process of 3D rendering the environments we see in the SiteSeer demo?

The 3D scans were made using a photogrammetry application which creates 3D meshes from a series of photos of an object. Natural features work well because there is a wealth of detail to use in the reconstruction, and any resulting artifacts are disguised by the resulting mesh, or even augment it.

What has been the biggest challenge in building this project demo?

The most difficult part was lining up the 3D elements with 2D photos or footage shot of the same feature. The 3D rendering was done in Maya, but there is no inherent land-marking or positioning system. Everything was lined up manually within the 3D environment using the features visible in the photos and the 3D mesh.

It was a lot of work for me to gather the photographic material at a location, and then assemble it in 3D space, but there are many technologies on the horizon that will automate much of this work. There are now panorama and hyperlapse apps and there will soon be 3D scanning mobile devices. SiteSeer is a system to join these various elements together in a reconstruction of a particular environment.

Why did you choose the medium of hyperlapse to show the different areas of land you then adapt into your 3D-rendered world?

The hyperlapse and steadicam sequences are a means of traveling seamlessly into and out of these locations, and then picking which new location to explore. I think hyperlapse sequences are a great mix of timelapse and camera motion and they quickly impart a good geographical sense of the environment where they were shot, so they mix well with the 3D elements.

How did the animation of your previous project, Fabricated, influence your technique for SiteSeer?

My other project, a 19-minute stop-motion animated opus entitled Fabricated, is quite different from this project on the surface, but there are many ways these projects have fed off of each other. Both are filmic environments that are manufactured or reconstructed from found elements, and both are journeys of exploration. I taught myself Maya as a fun break from the extreme tedium of stop-motion, if you can believe that. And hiking to beautiful remote locations with a camera was a fun break from both of these workflows.

How do you imagine people might use SiteSeer in their daily lives?

We are taking more photos than ever before, but looking at them less and less. I think that photo browsing software has not caught up with the way we are using cameras now, nor fully utilizing the information contained in the images themselves.

It is still a very big step before SiteSeer could be realized as an actual software package, but assembling this video helped to solidify the design and to demonstrate the possibilities of this system.