Using Augmented Reality in Facility Management – more potential than challenges?
14.11.2020 – At Granlund, we are tapping into the potential of Augmented Reality for facility management. Our goal is not only to develop an application for a specific building or built environment, but also to create a universal solution that can be easily applied to any building.
X-Reality including VR (Virtual Reality) and AR (Augmented Reality) is such an emerging technology that a lot of researchers and companies are studying its impacts on several traditional industries. In the fields of architecture, engineering and construction, ready-made VR solutions are already available for designers to preview their work in the design phase. It’s also possible to find AR solutions for design previews or quality checks in the construction phase. However, it is relatively rare to see an application made for the maintenance phase, especially for facility management.
Figure 1. Visualization of HVAC models in AR
First step: Visualizing the HVAC facilities
In the beginning of our Augmented Reality development, we started with a very simple feature: visualizing the HVAC facilities. There are several ways to achieve this.
AR markers are the oldest, simplest, but the most reliable approach to initializing AR. They are usually images with a unique rich feature, that can be recognized by computer vision. The weakness of AR markers is that they need to be attached everywhere in the building. This is fine for the construction phase but not practical for future building maintenance.
Manual adjustment is another way for how some existing AR solutions initialize AR. This approach can be considered as an alternative of markers, and it can be found in several existing solution, such as GAMMA AR, Trimble Connect and DaluxFM. It does not require any markers but requires users to perform some manual steps to adjust the pose of the model. Since it doesn’t require markers, it can be easily applied to any environment. But every time the user moves to another location, the pose of the model must be calibrated manually.
Figure 2. Manual adjustment to initialize AR (GAMMA AR)
The main-stream marker-less approach is based on ‘point-cloud’, which is a collection of feature points in the physical environment. The point-cloud of a certain object (e.g. a desk) or area (e.g. a corner) can be collected by taking pictures or videos. The calculated point-cloud is then saved locally or to the cloud. As the user opens the camera, what is in the cache can be matched with the saved point-cloud and the location can be recognized.
Moving on: Finding solutions for scale, reliability, and content
The visualization can be easily achieved with existing tools today. But visualization is not enough for a facility management application. We need to support users moving around in the building while making sure the pose of the whole model is still accurate enough, just as reliable, and stable as a real object. We need scale and the bigger the scale, the better.
For facility management, ideally, room-scale AR tracking is not enough. Instead, we need ‘building-scale’ tracking, as the maintenance engineers have the need to move around within the whole building, according to the service requests or maintenance tasks.
Larger scale is still challenging to be achieved. The building scale or even planet-scale AR tracking is a hot topic that a lot of researchers and startups are working on. The ideal approach is to generate a full point-cloud of the whole building. If we have a complete point-cloud of every single corner of the physical building, users can ‘locate’ no matter how they move in the building.
Another practical approach can be to deploy AR anchors for certain points-of-interest in the building. The user is located based on the network of the anchors in the building. When they move around, they automatically get re-localized according to the nearest anchor. Based on a reliable positioning, we can add AR content, such as HVAC models, and make sure they are always in correct pose no matter how the user moves in the building.
Figure 3. The approach of deploying multiple ‘anchors’ in the building for locating.
Since we are building an industrial software instead of a game for fun, reliability is crucial. Regarding reliability, there are two challenges we face – lost tracking and sensor drift.
When users are moving in the building, motion tracking is essential. If the tracking is lost, meaning user’s movement is not tracked, and the ‘internal map’ maintained by the AR session is distorted, as a result, the AR content could be in a wrong pose.
Figure 4. Example of tracking lost
Sensor drift is a serious problem
Since the motion tracking is based on sensors like gyroscope and accelerometer, the sensors can get effected by heavy motion, such as shaking, or when the phone is dropped to the ground. Unfortunately, sensor drift cannot be avoided if a gyroscope is being used in motion tracking. But it’s possible that we automatically re-localize from time to time when the user walks from one anchor to another, so that the AR content can be ‘calibrated’.
Figure 5. Example of sensor drift
The challenges regarding the Augmented Reality content are mainly about the model. Since the AR content here is the original IFC model from designers, the amount of content is usually massive. We need not only the geometry data but also the Building Information Model (BIM) data. Therefore, at least a cloud-based model service with optimization is crucial, and based on that, a dynamic, flexible component loading would be beneficial.
Pursuing unlocked potential
Despite challenges, at Granlund we are working hard to build a generic building-scale AR solution. We believe that Augmented Reality has a lot potential to be explored and utilized. With the swift development of hardware and AR engines, one day the virtual AR content will be an integral part of the building and it can be used to benefit not only the maintenance engineers but anyone in the building.
Mao Lin
Mao Lin works as an XR Developer in Innovation and development at Granlund.
Want to hear more?
Mao Lin
More blog posts
Development of sustainable solutions expands from real estate to city blocks
06.02.2024 – At the moment, all development activities are focused around mitigating climate change. We must find new ways of reducing emissions.
Three things to consider when choosing a data center designer
20.11.2023 – Responding to the ever-growing need for sustainable data processing means new data center sites are popping up and existing ones…
Why wouldn’t you share data?
25.10.2023 – The lack of a data sharing culture prevents the construction and real estate industry from utilising data to its full…
How we calculate our carbon handprint
10.10.2023 – When we help our customers reduce their carbon footprint, we increase our carbon handprint. We calculate our carbon handprint using…
What’s data got to do with it?
28.09.2023 – Imagine a building project where a building information model could simulate and analyse different scenarios based on energy, costs, schedules,…
Should we calculate carbon handprint alongside carbon footprint?
30.03.2023 – Carbon handprint aims at capturing the positive climate impacts that a company produces to its’ clients or to the society…
EU taxonomy for sustainable activities should not be ignored
31.01.2023 – The EU classification system for sustainable funding, also known as EU taxonomy, impacts construction projects. The planning stages of a…
Recycling waste heat from data centres to district heating networks should be increased
11.01.2023 – Energy-efficiently built data centres are part of the power reserve of the electricity grid. By using renewable energy sources for…
How data center operators should prepare for water risks?
17.12.2021 – Data centers are long-term investments and the clients appreciate security, connectivity, stability and scalability. How should the data center operators…
Smart matters! 4 essential questions to make sure your building is smart
07.12.2021 – As part of the Finnish team in the Dubai 2020 Expo, we at Granlund are excited to talk about our…
