Using Augmented Reality in Facility Management – more potential than challenges?
14.11.2020 – At Granlund, we are tapping into the potential of Augmented Reality for facility management. Our goal is not only to develop an application for a specific building or built environment, but also to create a universal solution that can be easily applied to any building.
X-Reality including VR (Virtual Reality) and AR (Augmented Reality) is such an emerging technology that a lot of researchers and companies are studying its impacts on several traditional industries. In the fields of architecture, engineering and construction, ready-made VR solutions are already available for designers to preview their work in the design phase. It’s also possible to find AR solutions for design previews or quality checks in the construction phase. However, it is relatively rare to see an application made for the maintenance phase, especially for facility management.
Figure 1. Visualization of HVAC models in AR
First step: Visualizing the HVAC facilities
In the beginning of our Augmented Reality development, we started with a very simple feature: visualizing the HVAC facilities. There are several ways to achieve this.
AR markers are the oldest, simplest, but the most reliable approach to initializing AR. They are usually images with a unique rich feature, that can be recognized by computer vision. The weakness of AR markers is that they need to be attached everywhere in the building. This is fine for the construction phase but not practical for future building maintenance.
Manual adjustment is another way for how some existing AR solutions initialize AR. This approach can be considered as an alternative of markers, and it can be found in several existing solution, such as GAMMA AR, Trimble Connect and DaluxFM. It does not require any markers but requires users to perform some manual steps to adjust the pose of the model. Since it doesn’t require markers, it can be easily applied to any environment. But every time the user moves to another location, the pose of the model must be calibrated manually.
Figure 2. Manual adjustment to initialize AR (GAMMA AR)
The main-stream marker-less approach is based on ‘point-cloud’, which is a collection of feature points in the physical environment. The point-cloud of a certain object (e.g. a desk) or area (e.g. a corner) can be collected by taking pictures or videos. The calculated point-cloud is then saved locally or to the cloud. As the user opens the camera, what is in the cache can be matched with the saved point-cloud and the location can be recognized.
Moving on: Finding solutions for scale, reliability, and content
The visualization can be easily achieved with existing tools today. But visualization is not enough for a facility management application. We need to support users moving around in the building while making sure the pose of the whole model is still accurate enough, just as reliable, and stable as a real object. We need scale and the bigger the scale, the better.
For facility management, ideally, room-scale AR tracking is not enough. Instead, we need ‘building-scale’ tracking, as the maintenance engineers have the need to move around within the whole building, according to the service requests or maintenance tasks.
Larger scale is still challenging to be achieved. The building scale or even planet-scale AR tracking is a hot topic that a lot of researchers and startups are working on. The ideal approach is to generate a full point-cloud of the whole building. If we have a complete point-cloud of every single corner of the physical building, users can ‘locate’ no matter how they move in the building.
Another practical approach can be to deploy AR anchors for certain points-of-interest in the building. The user is located based on the network of the anchors in the building. When they move around, they automatically get re-localized according to the nearest anchor. Based on a reliable positioning, we can add AR content, such as HVAC models, and make sure they are always in correct pose no matter how the user moves in the building.
Figure 3. The approach of deploying multiple ‘anchors’ in the building for locating.
Since we are building an industrial software instead of a game for fun, reliability is crucial. Regarding reliability, there are two challenges we face – lost tracking and sensor drift.
When users are moving in the building, motion tracking is essential. If the tracking is lost, meaning user’s movement is not tracked, and the ‘internal map’ maintained by the AR session is distorted, as a result, the AR content could be in a wrong pose.
Figure 4. Example of tracking lost
Sensor drift is a serious problem
Since the motion tracking is based on sensors like gyroscope and accelerometer, the sensors can get effected by heavy motion, such as shaking, or when the phone is dropped to the ground. Unfortunately, sensor drift cannot be avoided if a gyroscope is being used in motion tracking. But it’s possible that we automatically re-localize from time to time when the user walks from one anchor to another, so that the AR content can be ‘calibrated’.
Figure 5. Example of sensor drift
The challenges regarding the Augmented Reality content are mainly about the model. Since the AR content here is the original IFC model from designers, the amount of content is usually massive. We need not only the geometry data but also the Building Information Model (BIM) data. Therefore, at least a cloud-based model service with optimization is crucial, and based on that, a dynamic, flexible component loading would be beneficial.
Pursuing unlocked potential
Despite challenges, at Granlund we are working hard to build a generic building-scale AR solution. We believe that Augmented Reality has a lot potential to be explored and utilized. With the swift development of hardware and AR engines, one day the virtual AR content will be an integral part of the building and it can be used to benefit not only the maintenance engineers but anyone in the building.
Mao Lin
Mao Lin works as an XR Developer in Innovation and development at Granlund.
Want to hear more?
Mao Lin
More blog posts
Teknologiasta tukea CSRD-vaatimusten täyttämiseen
21.10.2024 – Kestävyysasioista vastaava ammattilainen: älä jätä tätä herkullista hetkeä hyödyntämättä! Käytä raportointivaatimuksia mandaattinasi rakentaa kestävyysmittaristosi ja seurantasi sellaiseen iskuun, että siitä…
Vuosikymmenet tikittänyt korjausvelka on mahdollista saada haltuun
26.08.2024 – Rahat ovat kunnissa ja seurakunnissa aina vain tiukemmassa, ja samaan aikaan korjausvelka pitäisi saada haltuun. Haluan rohkaista tekemään asioita uudella…
Miten tehdä edukseen erottautuvaa vastuullisuusraportointia?
21.08.2024 – Tuntuuko sinustakin, että eri toimijoiden vastuullisuusraportit ovat kuin toistensa kopioita? Liian usein ne tosiaan ovat sitä.
Vastuullisuudella voi olla ratkaiseva merkitys kiinteistökaupassa
14.08.2024 – Jos jotain voin ennustaa tässä muuttuvassa maailmassa, niin sanon, että vastuullisuudella on aivan selvästi vaikutusta kiinteistön myynnin helppouteen.
Onko käytönaikaisille ympäristösertifikaateille vielä tarvetta?
06.08.2024 – Meiltä kysytään usein, miten sertifikaattien käy tulevaisuudessa. Kannattaako niitä yhtä teettä kiinteistökohteisiin vai onko tilalle tulossa jokin parempi tapa todentaa…
Löperöistä hiilineutraaliusväittämistä pitää hankkiutua eroon
01.08.2024 – Välillä innostus ehtii uskottavuuden edelle. Nyt on hyvä aika tarkistaa, kestävätkö oman yrityksen hiilineutraaliusväittämät lähempää tarkastelua.
Parhaat sijainnit datakeskuksille löytyvät Suomesta ja Ruotsista
11.06.2024 – Kaikista maailman maista datakeskuksen sijaintipaikaksi kannattaa valita Suomi tai Ruotsi. Meillä on monta kilpailuvalttia, joilla päihitämme kilpailijat esimerkiksi Keski-Euroopassa.
Five Key Insights into the Future Superblock – Achieving Carbon Neutral Urban Planning
11.06.2024 – Local area energy plans support carbon neutral urban planning. The superblock of the future is built on these principles.
Viisi oivallusta tulevaisuuden superkortteleista vievät kohti hiilineutraalia kaupunkisuunnittelua
11.06.2024 – Alue-energiajärjestelmä tukee hiilineutraalia kaupunkisuunnittelua. Tulevaisuuden superkortteli rakentuu näiden oivallusten varaan.
Jopa kolmannes luontokadosta johtuu rakentamisesta
07.06.2024 – Vastuulliset rakennusalan yritykset ovat jo ottaneet käyttöön luontovaikutuksia arvioivia mittareita. Tämä vaikuttaa koko rakentamisen arvoketjuun ja pakottaa kaikki muutokseen.
