In this interview, we dive into the fascinating world of Augmented Reality (AR) and technology up-skilling with Dawid Swatowski, Senior Software Engineer at Holisticon Connect and a key AR Up-Skilling project member. Dawid’s valuable insights shed light on the innovative applications of AR technology and the challenges faced during the project. It’s evident that his acquired skills have the potential to shape the future of interactive experiences, and this interview provides a glimpse into his journey towards up-skilling in AR.

Getting Started with AR

HOLISTICON CONNECT: Please tell us about your exciting journey with the Team Up-Skilling project.

Dawid Swatowski: We initiated work on an internal project to create a virtual presentation of a coffee machine in our company’s kitchen. We aimed to use Augmented Reality (AR) technology to display, recognize, and interact with this device. We wanted to enable users to navigate the coffee machine’s virtual space while maintaining its elements’ continuity and allowing interaction through touch, which would trigger visual functions and display descriptions.

Overcoming Challenges

HOLISTICON CONNECT: What challenges did you encounter along the way, and what initial hurdles did you have to overcome at the beginning of the project?

Dawid: Initially, we had to learn AR technology and tools, which was challenging because we had no prior experience in this area, but instead, we had a lot of experience in C++. Based on that, we chose Unreal Engine as the project tool, one of the most popular engines for creating AR applications. However, we had certain technical requirements, such as the app running on Android and performing live image analysis for coffee machine recognition. We used the OpenCV library for image analysis because it’s an open-source library available in C++, which made it easier to integrate with Unreal Engine, allowing code writing in C++. It was a starting point for our team.

Choosing the Right Tech Stack

HOLISTICON CONNECT: Why did you choose Unreal Engine as your project tool?

Dawid: Some of us were interested in game development, and Unreal Engine was one of the two most popular engines for creating games and AR applications. Additionally, Unreal Engine allows us to use C++, the base technology for all of us, making it easier to start our adventure with Augmented Reality.

HOLISTICON CONNECT: What technical challenges did you face during the project?

Dawid: By definition, we had to use Android-based mobile devices, and at the very beginning, we had an issue choosing the right hardware because not all Android devices have ARCore, which is necessary to handle AR. We had to adapt to owned devices and give up Android emulators as they didn’t support the required architectures. Furthermore, the configuration needed to connect Unreal Engine, Android, and OpenCV provided many challenges.

Skills Gained and Their Future Applications

HOLISTICON CONNECT: What skills did you acquire during the project, and how can these skills be useful in the future?

Dawid: Most of all, we learned how to handle AR technology, use tools like Unreal Engine, and integrate them with additional libraries like OpenCV. Throughout the project, we gained the skills needed to create mobile applications for Android and learned about the challenges associated with this technology. These skills have various applications. For example, creating interactive product presentations in Augmented Reality for customers or using AR for marketing purposes is possible. Moreover, the acquired skills can be valuable for developing games or educational applications based on AR. Learning live image recognition and analysis can also be useful in fields related to computer vision.

User Interaction Challenges

HOLISTICON CONNECT: Did you face any challenges related to user interaction, such as touchscreen or AR element handling?

Dawid: Yes, during the project, we had to address issues related to user interaction. Our AR elements were clickable, allowing users to trigger visual functions and display descriptions, so we had to learn how to identify the objects the user pointed to and solve problems related to the usability of visible elements. In short, not all of our ideas were user-friendly. Another challenge was creating 3D models, which require ingenuity that can slightly cover the lack of talent. However, we were still learning, especially on aspects such as lighting, movement, and the appearance of complex elements, like various coffee cups.

Navigating the AR Space

HOLISTICON CONNECT: How did the placement of elements in the AR space go?

Dawid: We had some difficulties with placing elements in the right place in the AR space. Unreal Engine detects simultaneously many horizontal and vertical planes, which allows pinning elements on them. However, our target was to find only one proper vertical area most relevant to the coffee machine position in the real world. We had to employ various methods to eliminate erroneous planes and focus on those crucial to our project.

Challenges in Generating Virtual Objects

HOLISTICON CONNECT: What other challenges emerged during the process of generating virtual objects?

Dawid: A big challenge was detecting objects through live image analysis. In addition to defining how to distinguish our target object from the rest of the background, we had to face the problems of the environment in which it is located. I’m talking about lighting and reflections, which it generates on reflective surfaces and different types of artifacts nearby. We learned various methods of filtering the image and eliminating unwanted elements. It was a time-consuming process and required a lot of trial and error, but I must admit that, from my perspective, it was a very interesting and satisfying task.

Current and Future Use of AR

HOLISTICON CONNECT: How can the skills you’ve acquired be applied in the future?

Dawid: Certainly, in many areas where AR already exists today, such as games, marketing, education, or entertainment in general. We can use them to advance AR projects to create new user experiences. Augmented reality is also a step before Virtual Reality, which is also a very interesting topic. Moreover, image analysis also has many applications in industry and security. Working with an application dedicated to Android also allowed us to gain basic knowledge related to this technology.

HOLISTICON CONNECT: Can you provide some examples?

Dawid: For example, to provide virtual paths. In the application I am currently working on, I’m creating a virtual layout of our office where users, while using mobile devices, could get help exploring the office interior. The office floor plan has marked rooms and exits, and users can click on them to see paths that allow them to reach selected destinations or even display evacuation routes. So, it’s another stage that allows expanding skills and facing more challenges in Augmented Reality.

Applications Beyond Gaming

HOLISTICON CONNECT: Of the technologies you’re learning, where do they have significant applications in the market? You mentioned games. Is that mainly what’s being used?

Dawid: These technologies have various application areas. For example, Pokémon GO is one of the most recognizable examples of Augmented Reality (AR) technology usage. However, this technology is not limited to games.

HOLISTICON CONNECT: Where else can it be applied?

Dawid: Television uses AR in its information programs as an interesting visual addition when presenting data on charts or diagrams. Some games are built based on this technology and allow the user to experience a world in which virtual reality mixes with the real world, just like mentioned before Pokémon GO. Companies use AR for occupational safety training, employee training, and warehouses, where it helps employees find their way to specific locations or items. It can also be utilized in museums, for instance, to create guide applications that allow visitors to obtain additional information about exhibits. In general, AR technologies can find applications in various fields, and the only limitation is one’s imagination.

HOLISTICON CONNECT: Alright, you know what, this sounds fascinating. Thank you for the conversation and for sharing your experience.

Dawid: Many thanks for the conversation and the opportunity to share our project and experience.

Passion And Execution

Who We Are

At Holisticon Connect, our core values of Passion and Execution drive us toward a Promising Future. We are a hands-on tech company that places people at the centre of everything we do. Specializing in Custom Software Development, Cloud and Operations, Bespoke Data Visualisations, Engineering & Embedded services, we build trust through our promise to deliver and a no-drama approach. We are committed to delivering reliable and effective solutions, ensuring our clients can count on us to meet their needs with integrity and excellence. 

more about Holisticon Connect