For the past few months, the pandemic has forced us to isolate from each other, transforming our private spaces into schools and workplaces. The latter being enhanced by the use of virtual spaces like Zoom. And though the mass virtualization of these spaces seems to be decades away, we need to consider the applicability of such technologies within our current context. This project was built on a simple premise: can we create meaningful connections with our most cherished physical spaces remotely?
With this in mind I started to explore the idea of making the physical virtual. Having approached this project as an opportunity to also learn about Virtual Reality and how people are likely to interact and experience it, I started wondering about the implications of digitally recreating a real classroom would mean. I decided to focus on our own classroom which we had temporarily shuttered due to the pandemic. I wanted to see if an accurate recreation and the interaction that would follow would mean/feel like anything.
Since this was all new to me, most of my research focused on finding and learning the tools that would help me complete this project.
Photogrammetry
When done well, photogrammetry is one of the most useful tools in digitizing the real world, and it’s been done at different scales. I learned how to use Meshroom, an open-source software, by using a video I had on my phone as a test. As is the case with a lot of these tools, a newer and more functional application has now hit the market (Display.land), making this often tedious process almost seamless. Using images and photos taken in the classroom, I attempted to recreate the objects and the dimensions of the space using photogrammetry. This was a difficult process as my images and videos were not compatible with Meshroom’s capabilities, and since we couldn’t access the physical space anymore… I decided I needed to explore the world of 3D modeling.
3D modeling: Blender
It all started with a donut…Yes, that was the first 3D model that I completed in Blender, a free and open-source 3D computer graphics software toolset used for creating animated films, visual effects, art, 3D printed models, motion graphics, interactive 3D applications, and computer games.. This was no coincidence. As it’s come to be known in Blender circles, local Australian hero Andrew Price or Blender Guru on Youtube tutorials has onboarded a lot of beginners, and as a result, many have started their journey with a donut. Learning Blender basics was an exciting phase, and being familiar with graphics and video editing software, I was able to navigate the many online tutorials with relative ease. Having completed the popular “donut” tutorial, I felt confident enough to at least begin building my own objects and environments. After countless hours of modeling, sculpting, texturing, adding materials and a few renders later, I had a working prototype. I had created a 3D model of the classroom.
Game engine: Unity
Learning the basics of Unity, and then integrating the Oculus Quest was frustrating…until it was done. This required the support of tutorials, and even more forum question/answer readouts. I was left very thankful for the online communities that continue to make their knowledge accessible, and on a side note, I would like to leave a similar open trail when completed with current projects and will be adding a tutorials section on this website (in the meantime, feel free to reach out if you have questions.) Adding character movement, working on in-scene cameras, and finally being able to integrate Oculus tools and walk around in the virtual classroom was exhilarating (and left me a bit nauseous at times.) For a first person view of this test, please see the video file below.
While I enjoyed the experience of navigating inside the virtual classroom, the fact that an Oculus Quest would have to be used to do so became an issue. I was trying to share this experience with my cohort members and though their early reactions to the recorded version of this walk through was positive, I knew that lack of accessibility to the device would continue to be a barrier. Because of this, I decided to take an additional step and transform the experience into what I believe eventually became a more meaningful one. By using Vuforia, a tool to add Augmented Reality capabilities inside of Unity, I created a marker based AR object of the classroom. Vuforia itself is a well integrated plugin which required just a few additional steps to complete.
So What?
What I appreciated the most on a practical level was the ability to create virtual duplicates or physical objects as a way into Digital Twins. Having a strong interest in looking at Cyber-Physical Systems that are monitored and simulated through digital twins, I realized that the ability to create these virtual replicas in a physics enabling game engine like Unity means that I can ultimately begin to integrate data into the scenes. In fact, this has pushed me to continue developing my programming skills with the hope that I can begin to add C# as a complement to Python since I want to integrate real time data such as location based weather, or even sensor based data.
As for the actual experience itself, I'm left wondering whether VR experiences as means of allowing people to visit real places, be they as tourist sites or else, will remain gimmicky. I found the AR outcome to be more profound than the virtual tour of the classroom, which begs the question of whether the former might not serve our need to remember and build emotional bonds with remote places better. In other words, detached from the people around it, a place is best captured by a souvenir, and not a virtual visit.